Search Engines

Authoritative, Generative, and Social Search

Something I have personally been thinking about for some time is that there are three distinct modes for organizing and finding resources and answers.


Aaron's citation of the Dewey system is that of an authoritative taxonomy. The pros of these are that they are authoritative, relatively stable, and efficient at getting you somewhat near to the resources you want. They also work well as filtering methods if you are familiar with the taxonomies. The cons are (a) they are static and rarely keep up with the modern pace of change (b) expensive if you want professionals to do the classification and questionable in the minds of authorities if you crowd source the classifications and (c) exclusive in the sense that there is a relatively steep educational and learning barrier for their use.


The second organizational method is generative. This is what we experience with standard search engines. Behind the scenes technology is indexing, re-indexing, reclassifying etc. on a more or less continual basis, and as searches are submitted the searches themselves become data that is used to organize the corpus of resources. The pros of this are that it is flexible, fast and generally effective. The cons are that (a) it takes a lot of resources and research to get beyond simple term matching and page ranking (b) many methods fail for collections that are too small (not enough link data) or too big (too many results) and (c) it is not trivial (even today) to learn to use search engines effectively.


The third method is social. This is the modern equivalent of "if you want to know something, ask a friend" but facilitated by social media. To the extent that the results of inquiries can be captured and fed into Generative methods, there are some real possibilities here since people are better than computers at understanding context and recognizing semantic equivalents within their domain of experience. The cons are (a) answers are non-authoritative (b) managing social search at scale is a new field with a host of challenges and (c) I am not sure if we have enough experience to gauge its true effectiveness.


Having said all that, my suggestion is that a fundamental principle for any search system or knowledge management system is that all three methods should be used and that each one should be used and combined when and where appropriate. Hope that makes some sense!



3 votes
Idea No. 22