Citations of documents

From WICSA Conference Wiki

We (Paul Clements and Mary Shaw) are preparing a state-of-the-discipline paper on Software Architecture. In the process, we analyzed a sample of the literature to see what kinds of books and papers are commonly cited and when they were written.

We obtained, courtesy of the CiteSeer administrator, a complete CiteSeer search on citations of papers that have "software architecture" in the title. The dataset had about 5500 citations (ignoring self-citations) to about 750 distinct books and papers with a bit over 500 distinct lead authors.

We're not huge fans of detailed citation counting, but we do think that citation patterns can give indications of broad trends.

Here are the results.

Bear in mind that our sample includes only papers with "software architecture" in the title. In WICSA 2005, only 2 of the 15 papers satisfy this criterion, so our sample probably includes only 10-15% of the literature. It would therefore be incorrect to say that the kth paper on the list below is the "k-th most cited paper on software architecture"

The first item, Shaw&Garlan's Software Architecture book, is also on CiteSeer's list of 200 most-cited publications in computer science [1]. Although some of the more-cited papers may contain specific architectures (for example, for protocols), none of the other papers on this list is principally in the area of software architecture (so it arguably is the most-cited software architecture publication.

Here's the current draft of the interpretive paragraph for the paper:

"One way to see the growth of the field is to examine the rate at which earlier results serve as building blocks for subsequent results. A rough estimate is provided by citation counts for papers with "software architecture" in the title. We obtained the results of a full search for such papers in the CiteSeer database. We consolidated variant citations for papers (up to 20 variants in some cases!) and ignored self-citations, yielding a sample of about 5500 citations to about 750 books and papers. Virtually all of the cited papers were published in 1990 or later; 75% were published after 1994. There were steady increases in the number of citations for papers published from 1991 to 1996 and a sharp increase for papers published in 1998. The most widely-cited two dozen books and papers were published between 1991 and 2000. They include five books ([1, 2, 6, 23, 24], 1995 to 2000), four papers presenting surveys or models for the field ([3, 4, 12, 17], 1992 to 1997), six papers dealing with architecture for particular domains ([10, 13, 18, 19, 20, 21], 1991 to 1998), seven formalizations ([7, 8, 9, 11, 14, 15, 22], 1992 to 1996), and one paper each on an architectural description language [5] and an analysis technique [16]. The major changes in this pattern since a similar count in 2000 [2] are an increase in citations of formalizations and substantial turnover in the most-cited papers about architectures for specific domains. This indicator is based on the published literature, so it naturally reflects the first three phases of development. Imperfect though this estimate may be, it still indicates very substantial growth over the past decade or so and a balance between exploration of specific problems and development of generalizations and formalizations. Of the two dozen most commonly-cited papers of 2000, fourteen remain among the most commonly-cited papers in 2005 - an indication that the seminal sources have been identified."