We presented SEASR and Meandre with an introduction to Text Mining at a workshop sponsored by ICHASS (Institute for Computing in Humanities, Arts, and Social Science) on July 28, 2008. “This workshop, Information-Rich Environments for Research and Teaching, aims to give humanities, arts, and social science faculty, researchers and students intensive hands-on experience for improving the quality of their work through access to advanced computing infrastructures and applications.”

Also during this workshop, we held a hands on lab where attendees used the Meandre Workbench to perform some text analysis.

Text Mining Presentation

SEASR Presentation

SEASR/Meandre Hands on Presentation

The SEASR Team has created an alpha release of infrastructure and components. This release focused on the execution engine, development environments, components, documentation, and examples. The release includes:

• Meandre Manager - administration tool for users, flows, and components
• Meandre Infrastructure - execution environment
• Meandre Development Eclipse plug-in - tool for installing components
• Meandre Workbench - visual icon-based programming environment
• Meandre Zig-Zag - development scripting language
• Community Hub : Keyword Cloud – end-user gateway to published applications (flows)
• Component and flow repositories

Our goal with this release was to reach developers who want to create new component and begin application development with the SEASR environment. We encourage you to download and provide us feedback.

To download SEASR/Meandre, Components and Examples, click here.

The SEASR development team is hard at work preparing a release of SEASR technologies to date.  The release will include SEASR’s semantic-web, data-driven execution engine, Meandre, as well as components, flows, clients, and plug-ins that will help humanities scholars and programmers to make better and easier use of digital archives and humanities computing resources.  Check back for a release date and news!

SEASR co-PI Loretta Auvil will participate in the Mellon-funded Project Bamboo Workshop. With other higher education; museum and library; and organization, society, and agency leaders from across the U.S., she will attend the second session of The Planning Process & Understanding Arts and Humanities Scholarship workshop, which will be held from May 15-17, 2008 at the University of Chicago.

SEASR is twice mentioned in the Project Bamboo proposal, which sets as its goal formulating a strategic plan for enhancing the arts and humanities through the “development of shared technology services” (3). As one possible approach, the proposal recommends service-oriented architectures—such as SEASR’s—which emphasize ”being able to re-use and weave together loosely-coupled, discrete, specialized technology services that come from other providers and projects rather than building and managing all on one’s own.” The proposal goes on to say that “Critical to such an approach is the implementation of a web services framework. Such a framework is not a vertical application that focuses on a single in-depth function or a self-contained software tool used directly by a user, but rather a horizontally integrating set of technologies and set of core shared capabilities that enable the creation, aggregation, and reuse of services and resources among scholars, projects, and institutions” (15-16). The passage notes SEASR’s special strength in data analysis and mining tools.

In imagining a vision of the humanities researcher of the future and her work process, the Bamboo proposal turns to SEASR once again, envisioning a synthetic Bamboo composer that uses a visual programming environment similar to the one SEASR uses today in its workbench (20).

Loretta Auvil was invited to present the keynote address at the Text Mining Workshop 2008, which was held in conjunction with the Eighth SIAM International Conference on Data Mining (SDM 2008) in Atlanta, GA on April 26, 2008.  Her presentation title echoes SEASR’s identifying phrase, ”Engineering Knowledge for the Humanities.”

Presentation

Abstract

Over the last decade NCSA’s Automated Learning Group has innovated data mining technologies for industry, government, and the sciences. In the past few years, we have broadened our focus to include knowledge discovery in the humanities. My presentation will focus on how we are negotiating humanities computing’s special challenges for data mining and analysis. I will discuss our early collaborative projects, FeatureLens and Nora, and SEASR (Software Environment for the Advancement of Scholarly Research), the Andrew W. Mellon Foundation-funded project we are now leading. Each of these projects has developed technologies customized to meet specific needs of the digital humanities community. FeatureLens–an early MONK (Metadata Offer New Knowledge) application–uses the machine learning approach of frequent pattern mining to identify fuzzy repetition patterns in a data collection, and with no initial human input. Nora–a case study for eighteenth- and nineteenth-century British and American literature–uses predictive modeling techniques to classify documents, even given complex and notoriously indistinct expert classes such as sentimental fiction. SEASR is our most ambitious project yet, employing a semantic-based, service-oriented architecture to build software bridges that allow users to access data stored in disparate formats and on incompatible platforms and to provide an enhanced environment for workflow and data sharing. The essential infrastructure SEASR provides will advance the capabilities of projects like our partner, MONK, a digital environment designed to help humanities scholars discover and analyze patterns.

The SEASR and NEMA (Networked Environment for Music Analysis) teams have transformed a dynamic music classification explorer developed by IMIRSEL (The International Music Information Retrieval Systems Evaluation Laboratory) into a SEASR application that can be reused in whole or part by music researchers everywhere. Ira Fuchs–Vice President of Research in Information Technology for The Andrew W. Mellon Foundation (sponsor of SEASR and NEMA)–gave the “Son of Blinkie” (SoB) explorer its first demonstration on April 16^th.

INTRODUCING SON OF BLINKIE

Innovations in digital technologies have changed the ways we create, access, analyze, share, and consume information. But to realize their full potential, we need to re-evaluate digital information technologies to consider whether their methods are hold-outs from the age of print and, if so, what improved means we can devise. IMIRSEL’s SoB [1, 2], a dynamic classification explorer for musical digital library users and researchers, offers such an advance to the way in which we access and analyze music.

In the print collections and their digital descendents, information is retrieved through metadata, or descriptive labels, imposed upon it by librarians, editors, and domain experts. This metadata is used to generate tables of contents, subject indexes, and other searchable formats. Once determined, such labels and their associated epistemologies tend to become fixed and accepted as fact; they present a closed system of established knowledge rather than provide a virtual landscape that encourages exploration and enables discovery.In developing Son of Blinkie—affectionately named after the earlier, simpler “Blinkie Thing” [3]—the researchers at IMIRSEL have sought to bring leading machine learning methods to bear on the problem of how to make better use of the now digital nature of music collections. They have developed a means for searching music automatically, using its features of composition rather than imposed metadata as a guide. Not only does this automated method improve the speed and accuracy of information retrieval, but it promises to enrich our understanding of music and its classification.

Faced with a collection of music, we often accept that the labels imposed by past listeners are accurate and/or informative. But listeners may hold conflicting opinions about a piece, and the piece itself may defy reductive labeling. Through analyzing a piece using its own compositional features, machine learning can help us to understand whether a given piece is representative of a genre or mood as a whole or to certain compositional tendencies within it, tendencies that may change over time, by performer, or even by performance. What’s more, Son of Blinkie (SoB) advances earlier attempts to automate digital music collection retrieval and analysis.

Consider the traditional train-test approach to building, evaluating, and using machine-generated audio-based classifications (e.g., genre, mood, artist, etc.) for Music Digital Libraries (MDL). It’s useful in some contexts, but has two serious shortcomings. First, the classifications are monic (i.e., only one class label per piece). This monicity ignores the fact that most music comprises a mix of moods and/or genres, etc. Second, the classifications are static (i.e., one class label per song) even though pieces evolve through several moods and/or genre mixes over their play time. The SoB system offers a new and superior method of digital music exploration, engineered to overcome train-test shortcomings and better capture the dynamic nature of music. SoB provides users with the capacity for highly configurable real-time classification, visualization, and audition.

Another important advancement made with SoB is that the application operates within SEASR’s service-oriented architecture, taking the form of a series of reusable, open-source components managed by and executed as a shareable workflow from SEASR’s community hub. Not only can users run SoB against their own data sets– with SEASR’s assistance in accepting different input formats stored on different platforms–but they can also reuse and revise components and workflows to build their own music research applications.

SON OF BLINKIE IN ACTION

SoB works by extracting a stream of features from audio tracks and applying a set of pre-trained classification models to short windows (10 sec.) of these features to generate posterior probability distributions in real-time. The display of the classification probabilities is synchronized with the audio playback, empowering users to dynamically explore the effects and interactions of an infinite number of parameters involved in automatic music classification. SoB permits users to select an arbitrary number of classification models from the system’s ever-growing model library. Currently SoB’s model library comprises two classification “task” collections: mood and genre classifiers.

Above, we show a user simultaneously exploring the different real-time behaviors of mood classification models and genre classification models. Each model is making different predictions on this particular 5-second slice of the incoming, never-heard-before, song. The user can visualize the models’ prediction probability distributions, which can help the user better appreciate the potential “mixture” of moods present. The user can also listen to the synchronized audio to better understand the strengths/weaknesses of each model.

Below is a view that shows how data flows through the Son of Blinkie system, as it operates within SEASR (specifically, the semantic, web-driven dataflow execution environment portion of SEASR, which we have named Meandre). Each component represents one step in processing the data. The components run (and so process data) in the order established by the flow: from receiving the song filename and model filenames from the web application, to loading the audio and model data into memory, to extracting a variety of features from the song, to applying the model to the extracted features, to returning the predicted results to the SEASR community hub (a web application) for visualization. Every time a different song is selected, the web application executes this same flow.

REFERENCES

1. Funded by The Andrew W. Mellon Foundation and the National Science Foundation (Grant No. NSF IIS-0327371). Thanks to M. C. Jones and the SEASR team for their technical assistance.
2. IMIRSEL is directed by Dr. J. Stephen Downie, Graduate School of Library and Information Science (GSLIS), UIUC (jdownie@uiuc.edu). His Co-PIs on the Son of Blinkie system are Kris West, School of Computing Science, University of East Anglia and Xiao Hu, GSLIS, UIUC.
3. Downie, J.S., Ehmann, A.F., and Tcheng, D. 2005: Real-time genre classification for music digital libraries. JCDL’05, 337.
4. NEMA Website: http://nema.lis.uiuc.edu.
5. SEASR Website: http://www.seasr.org.