Humans use entry-level categories — the labels people naturally use to name objects — as the default level of abstraction when communicating about the visual world. For example, people say "dog" rather than "Labrador Retriever" or "animal." While large-scale visual recognition systems have made impressive progress on fine-grained classification tasks, they do not address the problem of predicting the names that humans would naturally use. In this paper, we develop the first computational models for predicting entry-level categories at large scale by combining visual recognition outputs with linguistic knowledge derived from web text corpora. Our models successfully predict the nouns that humans naturally associate with images and map fine-grained visual categories to entry-level concepts.

The study of entry-level categories dates back to the seminal work of Rosch et al. (1976) in cognitive psychology. Entry-level categories represent the most informative yet cognitively economical level of abstraction for categorizing objects. At this level, categories share the most common attributes, have the most distinct shapes, and are named most quickly by human subjects. Despite the fundamental importance of this concept, computational vision has largely ignored entry-level categorization, focusing instead on either basic object detection or increasingly fine-grained classification.

Current large-scale recognition systems, such as those built on ImageNet, can classify images into thousands of fine-grained categories. However, the output labels are often overly specific for human communication — e.g., predicting "Pembroke Welsh Corgi" when a user would simply say "dog." Conversely, top-level categories like "organism" are too abstract to be useful. We argue that bridging visual recognition with human-level naming requires explicitly modeling the entry-level of categorization, which depends on both visual distinctiveness and linguistic conventionality.

Research on object categorization hierarchies in vision has explored WordNet and ImageNet taxonomies for organizing visual concepts. Deng et al. (2012) proposed Hierarchy and Exclusion (HEX) graphs for reasoning about semantic relationships. Work on attribute-based recognition attempts to describe objects via shared properties, which relates to — but does not directly address — entry-level naming. In natural language processing, word frequency, familiarity, and corpus statistics have been used to assess the naturalness of category labels. Our work is the first to systematically combine visual recognition with linguistic analysis to predict entry-level naming at scale.

The visual component leverages predictions from large-scale image classifiers trained on ImageNet's hierarchical label structure. Given an input image, the system obtains probability distributions over fine-grained categories. These probabilities are then propagated up the WordNet hierarchy to compute aggregated confidence scores at every level of abstraction. For a node n in the hierarchy, the visual score is the sum of posterior probabilities of all descendant leaf categories. This propagation naturally produces higher scores for broader categories that encompass many fine-grained classes.

The linguistic component captures how natural or conventional a category name is in everyday language. The authors extract several features from large web text corpora: (1) word frequency — more common words tend to be used at the entry level; (2) word length — entry-level names are typically shorter; (3) familiarity scores — derived from psycholinguistic databases; (4) contextual diversity — entry-level words appear in more diverse contexts. These features are combined into a linguistic naturalness score that biases the system toward the abstraction level humans would prefer. The key insight is that entry-level categories are determined not only by visual similarity structure but also by linguistic conventionality.

The final prediction model combines visual and linguistic scores using either a product-of-experts formulation or a learned combination via logistic regression. For a given image, the system traverses the WordNet hierarchy from fine-grained leaves to the root, evaluating the combined score at each node. The predicted entry-level category is the node that maximizes the joint visual-linguistic score. The authors also propose a threshold-based method that selects the most specific category whose visual confidence exceeds a learned threshold, effectively balancing specificity against naturalness.

The authors evaluate their models on a large-scale dataset of images with human-provided entry-level labels collected via Amazon Mechanical Turk. Annotators were asked to name objects in images using the word they would naturally use. The evaluation covers over 1,500 ImageNet categories and demonstrates that the combined visual-linguistic model significantly outperforms baselines using only visual or only linguistic features. The learned model achieves accuracy improvements of 15-20% over a visual-only baseline in predicting human-preferred category names. Furthermore, qualitative analysis shows that the model correctly predicts "dog" instead of breed-specific labels and "car" instead of model-specific names, aligning well with human intuitions.

This paper presents the first large-scale computational models for predicting entry-level categories, bridging the gap between fine-grained visual recognition and human-level object naming. By combining visual recognition with linguistic analysis, we demonstrate that entry-level prediction requires modeling both visual structure and language conventionality. Our models have practical implications for image description generation, visual search, and human-computer interaction, where using natural category names enhances user experience. Future work includes extending the approach to context-dependent naming and cross-cultural variations in entry-level categories.