Michael
Donoser

Cross-modal recipe retrieval — matching food images to recipes and vice versa — is key for intelligent food discovery. This paper introduces a hierarchical recipe Transformer that attentively encodes individual recipe components (titles, ingredients, instructions), paired with a self-supervised loss that leverages semantic relationships within recipes. The approach enables training on both image-recipe and recipe-only samples, achieving state-of-the-art results on the Recipe1M dataset. Code and models released publicly via Amazon’s GitHub.

Interactive fashion retrieval lets users refine image search by modifying specific visual attributes such as color or sleeve type. Existing methods learn entangled embedding spaces, causing unintended changes when a single attribute is manipulated. This paper trains convolutional networks to learn separate, attribute-specific subspaces so that swapping one attribute leaves others unaffected. The unified model handles attribute manipulation retrieval, conditional similarity retrieval, and outfit complementary item retrieval, achieving state-of-the-art performance across all three tasks.

State-of-the-art image segmentation relies on hierarchical structures built from local feature cues analyzed via spectral methods, but both steps remain computational bottlenecks. This paper shows that a discrete-continuous optimization of oriented gradient signals delivers segmentation performance competitive with the state-of-the-art on the BSDS 500 benchmark — without any spectral analysis — reducing computation time by a factor of 40 and memory requirements by a factor of 10.

Image-based localization matches query image interest points to a sparse 3D point cloud to recover camera pose. Standard approaches represent each 3D point by a single descriptor centroid, ignoring the full set of associated 2D descriptors. This paper reformulates matching as a discriminative classification problem using random ferns, which is memory- and runtime-efficient. An extension projects features into fern-specific embedding spaces, improving match rates and outperforming nearest-neighbor baselines.

Diffusion on affinity graphs propagates similarity information across a manifold to improve retrieval ranking. This paper provides a comprehensive revisit of diffusion-based retrieval, surveys the state-of-the-art, and derives a unified generic framework of which prior methods are special cases. Evaluating the framework across multiple retrieval tasks yields new algorithm variants; one achieves a perfect 100% bullseye score on the widely used MPEG-7 shape retrieval benchmark.

Object category localization is cast as a partial edge-contour matching problem against a single shape prototype, avoiding error-prone pre-processing such as contour decomposition or interest-point detection. All extracted edges participate in a partial contour matching step; matched fragments vote for location hypotheses via generalized Hough-style accumulation. The method yields competitive results on challenging benchmarks including ETHZ shapes and INRIA horses at low computational cost.

Affinity propagation clustering on local color and texture models identifies multiple salient image regions. Each salient region seeds a figure/ground segmentation obtained by minimizing a convex weighted total variation energy, guaranteeing a globally optimal binary solution per seed. The resulting redundant segmentations are merged into a single composite by analyzing local certainty across all solutions, producing coherent multi-region outputs without user initialization.

MSERs are powerful local feature detectors but are expensive to recompute from scratch each frame. This paper exploits the component tree underlying MSER detection to propagate and update region identities across video frames, reducing per-MSER computation by a factor of 4–10. A weighted feature vector and backward-tracking pass improve data association and robustness, demonstrated on license plate, face, and paper-fiber tracking with consistent speedups over frame-by-frame redetection.