Archive for the ‘Community Building’ Category

Loretta Auvil of the SEASR Team will be participating in a workshop at Colorado Learning and Teaching with Technology Conference (COLTT). The workshop takes place at the University of Boulder on August 13, 2009 from 1:30-3:45pm. The workshop will focus on SEASR Analytics for Zotero. You can register for the event at the COLTT website above.

This presentation will highlight the integration of two powerful tools–Zotero for data management and SEASR for analytics. Zotero was developed at the Center for History and New Media, George Mason University, and is a tool aimed at facilitating a user’s research process by providing mechanisms for collecting, managing, and citing Internet resources (websites, articles, books, etc.). Zotero functions as an extension of the popular open-source browser Firefox. One of the key features provided by Zotero is the ability to automatically extract metadata from online resources as part of the resource collection process and to store it conveniently. Zotero also provides advanced tagging and searching functionality, allowing the user to organize, find, and visualize the collected resources effortlessly.

SEASR provides a semantic-enabled web-driven dataflow-execution environment that allows others to create their own analytical components. The initial analytics are meant to be demonstrations to show capabilities such as tag cloud generation, sentence summarization, entity extraction, and citation network analysis of the selected data assets. Additional text analysis capabilities are forthcoming. SEASR provides analytics to enhance scholars’ use of digital materials by helping them uncover hidden information and connections, supporting the study of assets from small patterns drawn from a single text or chunk of text to broader entity categories and relations across a million words or a million books. These analytics are also provided as a Firefox extension. This application allows researchers to use the SEASR analytical tools with their Zotero assets in a straightforward way.

Bernie Ács of the SEASR Team made a remote presentation for the NINES/18th Connect Workshop held in Dublin on July 15, 2009. The presentation included an overview of the SEASR project. Also in remote attendance were Loretta Auvil and Xavier Llorà of the SEASR Team.

The presentation is available here.

Loretta Auvil, Boris Capitanu, and Amit Kumar of the SEASR Team participate in THATCamp 2009 at George Mason University on June 27-29, 2009. We had the opportunity to have a session on SEASR Analytics. We also had the opportunity to discuss SEASR with many humanities researchers.

Loretta Auvil and Bernie Ács of the SEASR Team participated in the Digital Humanities 2009 Conference at the University of Maryland on June 22-25, 2009. We had a poster called “SEASR Integrates with Zotero to Provide Analytical Environment for Mashing up Other Analytical Tools” by Boris Capitanu, Xavier Llorà, Loretta Auvil, Michael Welge, and Bernie Ács. We also had the opportunity to have discussions with many humanities researchers.

The SEASR Team held a Follow-up SEASR Workshop on the Monday, June 22, 2009 of Digital Humanities 2009 week. Loretta Auvil and Bernie Ács presented updates to the SEASR project. We had presentations from Andrew Ashton of Brown University, Clare Lewellyn and Michael Krot of JSTOR, Anoop Kumar of Tufts (VUE), and Susan Schreibman of Digital Humanities Observatory.

The presentation materials are available at http://dev-tools.seasr.org/confluence/display/Outreach/June2009Follow-up.

Bernie Acs and Loretta Auvil of the SEASR Team participated in Bamboo Workshop 5 held June 17-19 in Washington DC. Attendees participated in discussions regarding the Bamboo proposal to The Andrew W. Mellon Foundation.

The University of Victoria’s Digital Humanities Summer Institute (DHSI) was held on June 8-12, 2009. Loretta Auvil and Boris Capitanu taught the course entitled “SEASR in Action: Data Analytics for Humanities Scholar”. The slides and course materials for this workshop are at http://dev-tools.seasr.org/confluence/display/Outreach/DHSI-SEASR.

We had 15 students registered for the course. The course covered the following topics: Overview of SEASR infrastructure (components, flows, applications), Introduction to text mining tools, and Using and creating Zotero flows.

Loretta Auvil and Xavier Llorà of the SEASR Team participated in Bamboo Workshop 2 held October 16-18, 2008 in San Francisco, CA. Attendees participated in discussions.

SEASR co-PI Loretta Auvil will participate in the Mellon-funded Project Bamboo Workshop. With other higher education; museum and library; and organization, society, and agency leaders from across the U.S., she will attend the second session of The Planning Process & Understanding Arts and Humanities Scholarship workshop, which will be held from May 15-17, 2008 at the University of Chicago.

SEASR is twice mentioned in the Project Bamboo proposal, which sets as its goal formulating a strategic plan for enhancing the arts and humanities through the “development of shared technology services” (3). As one possible approach, the proposal recommends service-oriented architectures—such as SEASR’s—which emphasize ”being able to re-use and weave together loosely-coupled, discrete, specialized technology services that come from other providers and projects rather than building and managing all on one’s own.” The proposal goes on to say that “Critical to such an approach is the implementation of a web services framework. Such a framework is not a vertical application that focuses on a single in-depth function or a self-contained software tool used directly by a user, but rather a horizontally integrating set of technologies and set of core shared capabilities that enable the creation, aggregation, and reuse of services and resources among scholars, projects, and institutions” (15-16). The passage notes SEASR’s special strength in data analysis and mining tools.

In imagining a vision of the humanities researcher of the future and her work process, the Bamboo proposal turns to SEASR once again, envisioning a synthetic Bamboo composer that uses a visual programming environment similar to the one SEASR uses today in its workbench (20).

The SEASR and NEMA (Networked Environment for Music Analysis) teams have transformed a dynamic music classification explorer developed by IMIRSEL (The International Music Information Retrieval Systems Evaluation Laboratory) into a SEASR application that can be reused in whole or part by music researchers everywhere. Ira Fuchs–Vice President of Research in Information Technology for The Andrew W. Mellon Foundation (sponsor of SEASR and NEMA)–gave the “Son of Blinkie” (SoB) explorer its first demonstration on April 16^th.

INTRODUCING SON OF BLINKIE

Innovations in digital technologies have changed the ways we create, access, analyze, share, and consume information. But to realize their full potential, we need to re-evaluate digital information technologies to consider whether their methods are hold-outs from the age of print and, if so, what improved means we can devise. IMIRSEL’s SoB [1, 2], a dynamic classification explorer for musical digital library users and researchers, offers such an advance to the way in which we access and analyze music.

In the print collections and their digital descendents, information is retrieved through metadata, or descriptive labels, imposed upon it by librarians, editors, and domain experts. This metadata is used to generate tables of contents, subject indexes, and other searchable formats. Once determined, such labels and their associated epistemologies tend to become fixed and accepted as fact; they present a closed system of established knowledge rather than provide a virtual landscape that encourages exploration and enables discovery.In developing Son of Blinkie—affectionately named after the earlier, simpler “Blinkie Thing” [3]—the researchers at IMIRSEL have sought to bring leading machine learning methods to bear on the problem of how to make better use of the now digital nature of music collections. They have developed a means for searching music automatically, using its features of composition rather than imposed metadata as a guide. Not only does this automated method improve the speed and accuracy of information retrieval, but it promises to enrich our understanding of music and its classification.

Faced with a collection of music, we often accept that the labels imposed by past listeners are accurate and/or informative. But listeners may hold conflicting opinions about a piece, and the piece itself may defy reductive labeling. Through analyzing a piece using its own compositional features, machine learning can help us to understand whether a given piece is representative of a genre or mood as a whole or to certain compositional tendencies within it, tendencies that may change over time, by performer, or even by performance. What’s more, Son of Blinkie (SoB) advances earlier attempts to automate digital music collection retrieval and analysis.

Consider the traditional train-test approach to building, evaluating, and using machine-generated audio-based classifications (e.g., genre, mood, artist, etc.) for Music Digital Libraries (MDL). It’s useful in some contexts, but has two serious shortcomings. First, the classifications are monic (i.e., only one class label per piece). This monicity ignores the fact that most music comprises a mix of moods and/or genres, etc. Second, the classifications are static (i.e., one class label per song) even though pieces evolve through several moods and/or genre mixes over their play time. The SoB system offers a new and superior method of digital music exploration, engineered to overcome train-test shortcomings and better capture the dynamic nature of music. SoB provides users with the capacity for highly configurable real-time classification, visualization, and audition.

Another important advancement made with SoB is that the application operates within SEASR’s service-oriented architecture, taking the form of a series of reusable, open-source components managed by and executed as a shareable workflow from SEASR’s community hub. Not only can users run SoB against their own data sets– with SEASR’s assistance in accepting different input formats stored on different platforms–but they can also reuse and revise components and workflows to build their own music research applications.

SON OF BLINKIE IN ACTION

SoB works by extracting a stream of features from audio tracks and applying a set of pre-trained classification models to short windows (10 sec.) of these features to generate posterior probability distributions in real-time. The display of the classification probabilities is synchronized with the audio playback, empowering users to dynamically explore the effects and interactions of an infinite number of parameters involved in automatic music classification. SoB permits users to select an arbitrary number of classification models from the system’s ever-growing model library. Currently SoB’s model library comprises two classification “task” collections: mood and genre classifiers.

Above, we show a user simultaneously exploring the different real-time behaviors of mood classification models and genre classification models. Each model is making different predictions on this particular 5-second slice of the incoming, never-heard-before, song. The user can visualize the models’ prediction probability distributions, which can help the user better appreciate the potential “mixture” of moods present. The user can also listen to the synchronized audio to better understand the strengths/weaknesses of each model.

Below is a view that shows how data flows through the Son of Blinkie system, as it operates within SEASR (specifically, the semantic, web-driven dataflow execution environment portion of SEASR, which we have named Meandre). Each component represents one step in processing the data. The components run (and so process data) in the order established by the flow: from receiving the song filename and model filenames from the web application, to loading the audio and model data into memory, to extracting a variety of features from the song, to applying the model to the extracted features, to returning the predicted results to the SEASR community hub (a web application) for visualization. Every time a different song is selected, the web application executes this same flow.

REFERENCES

1. Funded by The Andrew W. Mellon Foundation and the National Science Foundation (Grant No. NSF IIS-0327371). Thanks to M. C. Jones and the SEASR team for their technical assistance.
2. IMIRSEL is directed by Dr. J. Stephen Downie, Graduate School of Library and Information Science (GSLIS), UIUC (jdownie@uiuc.edu). His Co-PIs on the Son of Blinkie system are Kris West, School of Computing Science, University of East Anglia and Xiao Hu, GSLIS, UIUC.
3. Downie, J.S., Ehmann, A.F., and Tcheng, D. 2005: Real-time genre classification for music digital libraries. JCDL’05, 337.
4. NEMA Website: http://nema.lis.uiuc.edu.
5. SEASR Website: http://www.seasr.org.