After teaching legal information retrieval to first year law students for a few weeks, I started thinking about how we actually search for legal sources. Barbara Bintliff’s thoughts were very interesting in this regard:
Legal research no longer requires beginning with knowledge of the law because the emphasis of electronic research is on facts and keywords, not legal concepts. Research now is truly a mechanical process of entering factual words into a database or search engine and retrieving results. These research results appear to support the realists’ claims that law has no internal consistency.
Barbara Bintliff, Context and legal research, Law Library Journal, Volume 99, Issue 2, 2007, pp 249–266
I still believe that there is an inherent structure in law, though invisible and hidden. Or at least the quest for law as a coherent, linked and consistent framework of rules. As the trend for simple and fast search continues, however, we go straight for the treasure hunt, the easy picked keywords that come to mind. We expect search engines to suggest results for us, instead of trying to identify the most appropriate keywords first.
Thinking more about the concepts than the actual facts will be one of the main challenges of legal information retrieval in the next years. If human cannot translate concepts into keywords, maybe computers can?
See also the earlier quotes of the month:
Though there are different theories on why people search for information – suggesting uncertainty, satisfying a goal or simple curiosity – a gap in knowledge seems rather convincing in a legal setting. Assuming that lawyers generally search for legal sources in order to deal with a working task, the following statement is very persuasive:
Some descriptions of a […] task were more in the ‘puzzle piece’ mode, of filling a slot, but there was considerable indication of unspecified information need at the beginning of the process. A lawyer explained it this way: ‘You have an idea in your mind of what you are going to find out. You know there is a slot that needs to be filled and you know the name of that slot but you don’t know what goes into it’.
Carol C Kuhlthau, S L Tama; Information search process of lawyers: a call for ‘just for me’ information services; Journal of Documentation; 57:1; 2001; pp 25–43
What strikes me in this regard is that the lawyer is aware of the missing piece but does not know how to complete the puzzle. Identifying the slot is not the same as picking the appropriate search words. Keywords might not always successfully represent the missing puzzle piece. Maybe it would be more of value for a legal IR system to know the other pieces, in order to be able to fill the gap ……
context, here we are again ……..
[NB] See also the earlier quotes of the month:
Information overload is apparently not a new phenomena but was already noticed by Morris L Cohen, co-author of How to Find the Law (pdf), in 1968:
[…] the materials of our law seem to be marked by an accelerating birth rate, an almost non-existent mortality rate, and a serious resistance to contraception on the part of both judges and legislators.
Morris L Cohen; Research habits of lawyers; Jurimetrics; 9:1968; pp 187–188
Maybe the delete movement within privacy could offer a solution here, allowing for non-relevant legislation and case-law to be deleted as it becomes obsolete. For example, non-referring to a specific statute or case for several years could lead to its deletion or it being moved into the to-be-inspected archive folder maintained by a newly appointed Authority for Quality Assurance within Law – AQAL.
Spring cleaning in the repository of legal information retrieval – the legal sources – would not only lead to more coherence within the law but, above all, improve retrieval of legal sources and the quality of legal research.
[NB] See also the earlier quotes of the month:
Continuing on the subject of context …..
[…] the major different cognitive structures of individual users […] may be represented to intermediary mechanisms and IR systems, e.g. the actual work task or interest leading to a current cognitive state which may end up in a problem or uncertainty state for the actual user. These mental activities take place in the context of a domain(s) of epistemic, social or organisational nature which, in addition to influencing the current searcher in a ‘historical’ socio-semantic sense […] also constantly influences the authors of texts and systems design attitudes. The most simple form of a domain is an academic subject field, which is essentially a social construct represented by the collective cognitive structures of the individuals forming the field. Other forms of domains are, for example, industrial sectors, individual firms or organisations, or professional groupings, such as journalists. The horizontal interactive processes […] display communicative properties that are different from the vertical ones because of human interference […]
Peter Ingwersen; Cognitive perspectives of information retrieval interaction: elements of a cognitive IR theory; Journal of documentation; 52:1; 1996; pp 3-50 (pdf)
Interestingly enough, Ingwersen links context to cognitive structures which, inter alia, consist of the social surroundings of the user, such as her organisational and professional framework; as well as her current work tasks and interests. He suggests that these structures should be presented to and utilised by the IR system in order to improve retrieval.
Context, in other words, is more than taking into account previous search behaviours, be it my own or others. It rather means building a workflow-dependent framework for the user and gathering meta-data about her current information task, as opposed to purely relying on previous search experiences.
[NB] See also my earlier posts on Keywords vs context, Context and association, and Law as an App.
Reading a lot of material on the user perspective of information retrieval, I came across the following quote:
[…] despite experience with using general-purpose search engines, users may never discover strategies such as those known by the expert reference librarians. The knowledge to use such strategies needs to be explicitly taught. Furthermore, the decomposition and associated explication of the declarative and procedural components pin-points the knowledge that needs to be transmitted to users.
Marcia J Bates & Suresh K. Bhavnani, Separating the knowledge layers: Cognitive analysis of search knowledge through hierarchical goal decompositions, Proceedings of the American Society for Information Science and Technology, 2002, 39(1), pp. 204-13
Bates and Bhavnani, very interestingly, point out that information retrieval involves different skills and knowledges, both declarative knowledge and procedural knowledge and that these skills have to play together in order for users to search effectively and retrieve sources of high quality.
Despite the common conception that individuals who can use Google are also very good at searching, the article shows that this is not the case. In my opinion, the simplicity of search engines’ interface contributes to the misconception that search is easy. Just type in a few words and let the computers do the rest. Knowing a bit more about the technology behind, at least which factors are taken into consideration when deciding on a document’s relevance, might help educating users about this everyday task which nevertheless might be of great importance.
Visiting the Annual Nordic Conference in Legal Informatics in Rovaniemi, Finland, I started thinking about the progress that information retrieval technology has made the last 30 years, and remembered the following statement from 1985:
The user’s task is simply to find the right combination of search terms to retrieve all and only the relevant documents. However, we believe that users should not be asked to shoulder the ‘blame, and perhaps an analogy will indicate why. Suppose you ask a company to make a lock for you, and they oblige by providing a combination lock; but when you ask them for the combination to open the lock, they say that finding the correct combination is your problem, not theirs. Now, it is possible, in principle, to find the correct combination, but in practice it may be impossibly difficult to do so. A full-text retrieval system bears the burden of retrieval failure because it places the user in the position of having to find (in a relatively short time) an impossibly difficult combination of search terms. The person using a full-text retrieval system to find information on a relatively large database is in the same unenviable position as the individual looking for the combination to the lock.
Blair, D.C., and Maron, M.E., An evaluation of retrieval effectiveness for a full-text document-retrieval system, Communications of the ACM 28, 289-299, 1985; p 298
Strangely enough, I am not sure that we have come so much further since then. Though search algorithms may have become more advanced, users still have to employ key words in order to retrieve possible important information. Personalisation of search (taking into account the previous searches of a person) is not the same as putting information retrieval in context (considering the current situation of the user). A clean user interface is not the same as simplicity and effectiveness of search.
A growing amount of legal information is in itself not a bad thing. Though quantity is not to be confused with quality, more data increases the chances of finding information that is applicable to one’s question. The challenge is, however, how to access this information.
Wholly new forms of encyclopedias will appear, ready-made with a mesh of associative trails running through them, ready to be dropped into the memex and there amplified. The lawyer has at his touch the associated opinions and decisions of his whole experience, and of the experience of friends and authorities.
Vannevar Bush, As We May Think, Atlantic Magazine, July 1945, link
In my opinion, context will become more and more important, and my doctoral thesis is dealing with how to create context in a legal information retrieval environment. Vannevar Bush’s vision of associative trails guides us in the right direction.
Interesting to note that Bush already in 1945 considered social media and knowledge sharing, especially within the legal community, to be a valuable asset.
Let me know your thoughts!