Calvin

Dataless Model Selection with the Deep Frame Potential

by

Calvin Murdock, Simon Lucey

Abstract

Choosing a deep neural network architecture is a fundamental problem in applications that require balancing performance and parameter efficiency. Standard approaches rely on ad-hoc engineering or computationally expensive validation on a specific dataset. We instead attempt to quantify networks by their intrinsic capacity for unique and robust representations, enabling efficient architecture comparisons without requiring any data. Building upon theoretical connections between deep learning and sparse approximation, we propose the deep frame potential: a measure of coherence that is approximately related to representation stability but has minimizers that depend only on network structure. This provides a framework for jointly quantifying the contributions of architectural hyper-parameters such as depth, width, and skip connections. We validate its use as a criterion for model selection and demonstrate correlation with generalization error on a variety of common residual and densely connected network architectures.

Read more

One paper accepted to CVPR 2020 as an oral presentation!

by
Dataless Model Selection with the Deep Frame Potential

Calvin Murdock; Simon Lucey

Dataless Model Selection with the Deep Frame Potential

Conference on Computer Vision and Pattern Recognition (CVPR), 2020, (Oral Presentation).

Abstract | Links | BibTeX

One paper accepted to ECCV 2018!

by

Deep Component Analysis via Alternating Direction Neural Networks

Calvin Murdock; Ming-Fang Chang; Simon Lucey

Deep Component Analysis via Alternating Direction Neural Networks

European Conference on Computer Vision (ECCV), 2018.

Abstract | Links | BibTeX

New paper posted to arXiv!

by

Paper

Deep Component Analysis via Alternating Direction Neural Networks

Calvin Murdock; Ming-Fang Chang; Simon Lucey

Deep Component Analysis via Alternating Direction Neural Networks

European Conference on Computer Vision (ECCV), 2018.

Abstract | Links | BibTeX

Deep Component Analysis via Alternating Direction Neural Networks

by

Calvin Murdock, Ming-Fang Chang, Simon Lucey

Abstract

Despite a lack of theoretical understanding, deep neural networks have achieved unparalleled performance in a wide range of applications. On the other hand, shallow representation learning with component analysis is associated with rich intuition and theory, but smaller capacity often limits its usefulness. To bridge this gap, we introduce Deep Component Analysis (DeepCA), an expressive multilayer model formulation that enforces hierarchical structure through constraints on latent variables in each layer. For inference, we propose a differentiable optimization algorithm implemented using recurrent Alternating Direction Neural Networks (ADNNs) that enable parameter learning using standard backpropagation. By interpreting feed-forward networks as single-iteration approximations of inference in our model, we provide both a novel perspective for understanding them and a practical technique for constraining predictions with prior knowledge. Experimentally, we demonstrate performance improvements on a variety of tasks, including single-image depth prediction with sparse output constraints.

Read more