Listening to a Glacier on a Warm Summer Day

In June, CDA Director Ben Rubin and Data Artist Jer Thorpe completed Herald/Harbinger, a work of light, movement, and sound that unfurls from the south lobby of Calgary’s Brookfield Place into the plaza.

Brett Gilmour, © 2018

Heralding the ascent of Earth’s Anthropocene period, Herald / Harbinger incorporates a collection of data feeds to illustrate the interrelationship between human activity in Calgary and the natural system of the Bow Glacier in the Canadian Rockies, which exists in a perpetual state of physical transformation.

The narrative of the artwork originates a few hundred miles west of the city in the Canadian Rockies, where the Bow Glacier melts, cracks and shifts every few minutes with changing temperatures, existing in a perpetual state of physical transformation.in 2016 and 2017, the artists constructed a solar-powered seismic observatory at the edge of the glacier. There, sensors register the near constant shifting of the restless ice and feed this information in real-time data to the artwork’s servers.

Brett Gilmour, © 2018

The artwork itself is a permanent public installation that renders the glacier’s movements as audible icy rumblings and visible displacements of scan lines on an array of LED lights. Inlaid patterns on the granite plaza surface represent the constant forces pushing the Bow Glacier down from the Wapta Icefield toward the Bow lake.

“Climate Change is an abstraction. We read about it, but it’s happening too slowly for us to perceive, and so it’s hard to accept it as something that’s real and urgent. This artwork is an attempt to give a ‘voice’ to a specific glacier in the hopes of making climate change something we can hear and see — something that feels real.” – Ben Rubin

Brett Gilmour, © 2018
Brett Gilmour, © 2018

As Calgary’s day gets busier and traffic levels rise, the patterns in the artwork are interrupted by the patterns of the urban life, with aggregated data from pedestrians’ footsteps and traffic sampled at 14 different locations around the city sometimes intruding upon the glacial tempo. Vehicle traffic is displayed as seven ‘roadways’ on the piece’s seven LED fixtures. Herald/Harbinger creates an encounter between Alberta’s natural systems and the city’s restive human activity, establishing a kind of conversation between these two realms.

And yet for all of its complexity, the artwork’s presence remains subtle, blending discretely into its urban surroundings even as it invites curious passersby to pause, listen, and investigate.

The substance of public art, for me, lies in the process of people discovering the artwork, and so it’s not always necessary to call attention to it. When someone happens upon the work and discovers it for themselves, the experience becomes personal, and when the work takes on a special meaning for someone, that person is more likely to tell friends about it, and that process, I think, is more valuable than advertising or publicity. – Ben Rubin

‘Activating Museums’ Data for Research, Scholarship, and Public Engagement

A vast number of Digital Humanities projects have emerged in the last few decades that digitized museum collections, archives, and libraries and made new data types and sources possible. Interdisciplinary labs such as the medialab at Sciences Po focus on the development of new tools that cater to methods and research in the Humanities. They especially address the regressive gap that has become evident in the current tools needed to analyze, comprehend, and present these new digital sources. Provenance and translocation of cultural assets and the social, cultural, and economic mechanisms underlying the circulation of art is an emerging field of scholarship encompassing all humanities disciplines.

‘With the migration of cultural materials into networked environments, questions regarding the production, availability, validity, and stewardship of these materials present new challenges and opportunities for humanists in contrast with most traditional forms of scholarship, digital approaches are conspicuously collaborative and generative, even as they remain grounded in the traditions of humanistic inquiry, this changes the culture of humanities work as well as the questions  that can be asked of the materials and objects that comprise the humanistic corpus.’

Internationally researchers concerned with the ‘social, cultural, and economic mechanisms underlying the circulation of art’ work with object-based databases that describe, document and store information about objects, operations, movements and provenance of objects. The scholarship working with those databases is grounded foremost in disciplines of the Humanities and Social Sciences: Sociology of Art studying the social worlds that construct a discourse of art and aesthetics; closely related to it the Social History of Art concerned with the social contribution to the appearance of certain art forms and practices; Economics and Art Business exploring global economic flow and the influence of wealth and management strategies on art; Art Theory developing discourse on concepts relating to a philosophy of art and therefore closely related to curating and art criticism; and Art History with research in provenance, image analysis and museum studies.  

Although established methods for collecting, analysing and storing of various data sources differ with the methodological approach each researcher takes, it can be said that the need for new computational tools that allow the analysis, storage, sharing and understanding of the new digital data sources are needed. Digital data collections can not be studied without tools that are appropriate for the analysis of digital textual, pictorial and numeric databases. Borgman (2015) illustrates the current discourse on Big Data/Little Data and the associated methodological approaches in current research communities and identifies much like Anne Burdick (2012) “there exists not one method, but many” for digital data analysis in the Humanities.

The research project uses the infrastructure of digital databases produced with methods in the DH but will advance a primary focus in computer-aided research with the development of current web based software development standards. Only in the past 7 years researchers started to develop “standardized, native application programming interfaces (APIs) for providing graphics, animation, multimedia, and other advanced features.” This standardization allows researchers to develop tools and applications that can be used in multiple different browsers and on different devices. This is to show that the digitization of object based collections (museum collections, libraries and archives etc.) has an established history but nevertheless the development of tools to access, display and analyse these digitized collections is just at the beginning of current  possibilities due to the development of web standards across browsers and devices. The research group will bring this standard to the work with digital museum collections and in general with digital object based collection.

Museums produce phenomenal amounts of data.

They do so by tradition, research about the provenance, narrative, material, production and cultural embeddedness are attached to every piece in a museum collection. By digitizing we do not only consider pictorial surrogates for the works or economic value (acquisition prize, insurance value and other administrative costs that keep a work of art in a specific physical condition) but rather, researchers have produced knowledge, a history and institutional contextualization about works of art in institutions. The mode of shareability of knowledge and access to a corpus of information about works of art in a museum context is, at the moment, unsatisfying but inevitable.

Another development is the creation of a data capitalism that we find in a current museum context especially through applications such as Google’s Art and Culture face detection software in which the private company asks their users to trade a face match with museum portraits for their user data and facial images which is believe to be used to “train and improve the quality of their facial recognition AI technology.” In contrast to this research projects in the Humanities have the aim to find and produce solutions for accessibility to information dissemination of museum collections that work based on the research, methods and data structures of the institution.     

But, the advent of digitization and digital modes of exhibitions have only exacerbated the possible facets of the work of art. Researchers from all disciplines in the humanities analyse and explore object based databases with a variety of quantitative and qualitative methods and data analysis tools. The proposed project aims to produce new digital tools for interdisciplinary and mixed method research in the digital humanities, working with extensive data sets that allow the inclusion of a variety of methods, data types and analysis tools.

In its preliminary phases, the project between three institutions is interested in exploiting databases of cultural material to pave the way for new research tools in contemporary art history. The collaborating institutions, The Médialab, Sciences Po, Paris Translocation Cluster at the Technische Universität Berlin and the Center for Data Arts, The New School, New York City will organize data sprints that started in Paris in Fall 2017.

The Data Sprints

This methodology in the overview of methods in the social sciences was borrowed from the development of free software and has been adapted to the new constraints weigh on researchers venturing into the world of digital data. It takes the form of a data sprint, a form of data-centered workshop designed to deliver a better understanding of datasets and conducive to formulate research questions based on their complex exploration. It is the interdisciplinary development of new digital tools for the humanities, a sequence mixing in a short duration, typically over one week: data mining, their (re) shaping and production of descriptive statistics and data visualizations.

“Data-sprints are intensive research and coding workshops where participants coming from different academic and non-academic backgrounds convene physically to work together on a set of data and research questions.”

The six phases of a data sprint are 1) Posing research questions; 2) Operationalizing research questions into feasible digital methods projects; 3) Procuring and preparing datasets; 4) Writing and adapting code; 5) Designing data visualizations and interface and 6) Eliciting engagement and co-production of knowledge.

History of the project

The Medialab, Sciences Po, in coordination with the Centre national des arts plastiques (CNAP) and videomuseum (Consortium of modern and contemporary public art collections), has already tested the possibility of exploiting a data management infrastructure to tease out research questions in art history and sociology of art during a data sprint organized in September 2016 in Paris. This workshop produced an initial study of the Goûts de l’État – the tastes of the State, for once a better rhyme in English than in French – by exploiting the rich documentation of the acquisition and circulation of contemporary art (about 83,956 works in its Fonds national d’art contemporain or FNAC, managed by the CNAP) by the French State since the revolution. The experience has been positive for both the art historians participating and for the CNAP cadres who have also learned quickly about many aspects of their database from the new formats and visualizations tried by the programmers and designers. Several research projects emerged as a result of this encounter between art historians, programmers and designers.

Data Sprint at the MediaLab at Sciences Po in November 2017

Since April 2017, the researchers involved in the project are working together to connect with museums as partnering institutions to the project, strategize data sprints, and generate a team of researchers that will be invited to the ongoing data sprints. The MediaLab at Sciences Po organized the first data sprint in November 2017 inviting over 25 participants including museum staff of the Centre Pompidou. The participants gathered for one week at Sciences Po and developed over 50 visualizations and two working software prototypes based on their preliminary research and on site data analysis.

Research Questions in the First Data Sprint included:
  • Provenance and the translocation of cultural assets
  • Modalities and temporality of acquisition
  • Exhibition in the museum and circulation outside of the museum
  • Artistic groups and collectives in the museum
  • Social, cultural and historical embeddedness of the collection

Small working groups focused on at least on one of the following approaches:

Mixed-Method approaches

Qualitative and quantitative methods are used on the same issue and with the same priorities. Quantitative methods give insight into a data set and qualitative methods are used for single cases. Both the qualitative approach can lead to research questions that can be utilized for structuring the quantitative analysis and vice versa. Machine learning and quantitative methods are practiced as forms of ‘distant reading’ that allow the qualitative researcher a ‘deep dive’ with an understanding of patterns of the entire dataset.  

Machine Learning: Natural Language Processing

The method in natural language processing that will be focussed on in the workshop is ‘Named entity recognition (NER)’ to detect entities such as people, places or organizations in large text based data sets. NER will make it possible to map these entities according to geo locations, expressions of time or their category (attributes). After the first data sprint, a group of researchers begun to work on a forthcoming focus in image recognition as a tool for authentication in art history:

Machine Learning: Image recognition

In recent years the digitization of cultural collections (archives, museum collections, libraries etc.) has produced a massive amount of digital images. These surrogates for the material objects can be analyzed with qualities (shapes, color, theme, etc.) that the digital object has and can lead to categorization and the identification of patterns in large scale data sets.

The data that the partnering institution, Centre Pompidou in Paris, made available for the interdisciplinary team of researchers contains “over 100,000 works, the collections of the Centre Pompidou (musée national d’art moderne – centre de création industrielle) which make up one of the world’s leading references for art of the 20th and 21st centuries.

 

Dark Source: A Reflection of the Politics of Voting Technology

As the debate on the Midterm Elections of November 2018 gets more heated, senators from both parties have expressed serious concerns over threats to cybersecurity of electoral systems across different states. Several lists of recommendations to fortify the systems have been released at the state and federal levels by the Senate Intelligence Committee, the Secretary and Former Head of Homeland Security and the Federal Elections Commission emphasizing the need for Cyber-security Risk Assessments.
Between the record-number White House resignations and departures and the amorphous allocation of around $700 million in funding, with $380 million towards “rapidly” replacing aging election technology across the country and $307 million towards fighting potential cyber threats due to Russian interference, it is clear that the Trump Administration is not sufficiently prepared for the upcoming elections. These patterns of weak election technology and weaker cyber-security, however, are not a recent phenomenon.
In 2005, CDA Director Ben Rubin expressed through his art installation Dark Source, “the inner workings of a commercial electronic voting machine.”

The artwork presents over 2,000 pages of software code, a printout of 49,609 lines of C++ that constitute version 4.3.1 of the AccuVote-TS™ source code.

In Dark Source the code, which had been obtained freely over the internet following a 2002 security failure at Diebold, has been blacked out in its entirety in order to comply with trade secrecy laws.

In an essay subsequently published in Making Things Public: Atmospheres of Democracy, he elaborates on the complications with proprietary election technology in the context of 2004 elections.
We trust cash machines and gambling machines with our money, and we trust medical devices and autopilots with our safety, so why shouldn’t we also trust electronic voting machines with our ballots?
Proprietary voting technology, subject to no meaningful standards of security, reliability, or accuracy, is inherently vulnerable not only to malicious tampering but also to inadvertent failure.
Election systems must be returned to the public’s control, and one essential step will be to lift the veil of secrecy that cloaks the software.
As we continue to follow the trail of Election Security, here at Data Matters and raise the necessary concerns over the upcoming elections, it could be worthwhile to reflect upon the fallacies of the past.

Lifelong Analytics: Equity, Data Viz & Precision Medicine

About the Author: Emily Chu is a Data Visualization Developer and Motion Designer currently focusing on data visualization engineering, machine learning and interactivity. Her background spans program management, research, business, design and technology. (MS Data Visualization, Parsons School of Design) | Website: 3milychu.github.io

 

The Spotlight on Healthcare Data

The tidal wave of new healthcare technologies is dizzying. Telehealth, artificial intelligence, AR/VR, 3D printing and patient monitoring promise that the future of medicine will be more efficient, affordable, secure and personal. Most advancements spotlight healthcare data as the foundation: we are capturing it, sharing it, making use of it and disseminating it in ways we’ve never done before.

Healthcare Data’s Rising Value and Impending Inequity

Consider this year’s Economic Forum’s meeting in Davos, where key industry leaders stated that using global healthcare data available to them, Machine Learning will be able to uniquely pinpoint the most effective treatment for an individual. At the current rate of data representation, however, health systems will be much poorer at offering efficient and highly accurate treatment for individuals that are not of European or East-Asian descent.

Meanwhile, the momentum behind capturing healthcare information is the heightening awareness of its value and security. Companies like Nebula Genomics, for instance, are offering to sequence your genome and secure it on a blockchain, wherein you will have control over the transfer of this packet of information and it will only go where you send it. In a consumer-driven healthcare world, what types of customers will have the privilege to understand what this even means?

What we can do with healthcare data can level the playing field.

We can make it secure and affordable for everyone, regardless of condition, race, or socioeconomic background, to receive the most effective treatment available. Looking at the typical health system infrastructure, where do we start?

Enter Electronic Health Records

Electronic Health Records or Electronic Medical Record (EHR/EMRs) are now a standard method of healthcare data storage and exchange. Patients are able to view an electronic copy of their medical records and physicians are able to share test results with patients. It can be thought of as the start of healthcare data consumerization. It is perhaps the perfect training ground to help the most vulnerable populations understand –

  1. how valuable their healthcare data is and
  2. how to harness it to improve their health and receive the most affordable, effective treatments in the future.

Since its inception, we now know that approximately half of the U.S. population encounter difficulties in comprehending and utilizing their health information, ushering in the need for a “visual vocabulary of electronic medical information to improve health literacy”. In 2014, a study revealed that 63.9% of of EMR survey respondents complained that note-writing took longer, and as of 2017, 94% of physicians in a survey were overwhelmed by what they believe to be “useless data”.

Visualizing Healthcare Data for the Most Vulnerable: From Collection and Adoption to Accuracy and Feedback

One important step is to get the most vulnerable populations – lower literacy individuals, patients with chronic or debilitating conditions, the elderly – to find a real use in capturing data and finding an enjoyment in doing so. The following demonstrates an example of how this updated electronic health record might function.

From Integrated Treatment Adherence to Responsive Feedback to Lifelong Analytics

In Visual 1.0: Simple Gamification of Healthcare Activities (below), for example, the patient is first shown how medications and healthcare tasks such as “take your blood pressure” can be gamified in a simple user experience to encourage data collection.  

Visual 1.1: Progress Over Time (below) shows how collecting vitals and treatment plan adherence might then be synced and displayed in the record shared with physicians. 

In Visual 1.2 Breakout view of healthcare activity or Biometric Marker (below), consider that the main dashboard view can be broken down and analyzed easily by physicians.  

Visual 1.3 Condensed Progress Summary and Feedback for the Patient (below) then illustrates closing the feedback and health comprehension gap that is often left open after treatment, by condensing the analytics into a simple progress view over time. Recommendations for the medical aspect (i.e. treatment plans) or maintenance behaviors (i.e. exercise) are adaptive. For example, at predetermined check-in intervals or when tracking metrics trigger a certain threshold, the treatment plan adapts based on level of adherence or other care plans that were implemented. Finally, consider that patients should be able to view future states assigned to them by predictive analytics (not pictured). In this scenario, what I would call Lifelong Analytics, individuals securely own all their healthcare information and are able to compare how predictive analytic models place them in the future.

By using the electronic health record as a catalyst to drive data collection and adoption among the most vulnerable, we are securing a pool of representative information for groups that may otherwise be left behind in the race for precise treatment at near-to-no cost. Along the way, through digestible habits and user-friendly actions, patients will be exposed to the power behind documenting their healthcare information. Once individuals are empowered with their data and what it really means, we can imagine a future where people are quick to stand up for the ownership of their data – and ensure that advancements that are made considering their footprint.

Takeaways

The poor, the elderly, the sick and the underrepresented have much to offer to the future of medical practice. They offer interesting challenges and high payoffs in cost efficiencies. When we consider a future where data will be dynamically classified and trends predicted, it is important to concentrate adoption among these groups. Some methods we discussed in this article:

Making treatment plans easy to track and adaptable

Treatment plans should be easy to track. Monitoring can be easily integrated into our routines, or in the future – automatically reported back to us. Providers should be able to understand what adaptive measures need to be taken should we miss a dose, or life interferes with our rehabilitation plan.

Making our medical history secure, transparent and shareable

Technologies currently exist to ensure our healthcare information belongs to us, and we have ownership over where it is transferred virtually. Visualizing healthcare information using a visual vocabulary that demystifies our health history, and shared among all providers in our care network can strengthen this transparency.

From responsive feedback to lifelong analytics

Consider a future where individuals with secure ownership of their healthcare data can access not only responsive feedback from their care providers, but see how their lifelong analytics are affected with each stint of perfect treatment plan adherence or alternative care plan. In other words, imagine what predictive analytics has to say about us is eventually comprehensible and accessible to us as individuals.

By visualizing and making healthcare information for the most vulnerable readily accessible and comprehensible, we make it possible to access the most difficult treatments responsively and potentially risky treatments with transparency. In the end, this can teach an entire population how to better develop an infrastructure that prepares and cares for us when we too age, get sick or fall into disadvantaged circumstances.

Data Stories: What Space Oddity Looks Like

In the land of popular music, there has been little scarcity of fashion experiments. And David Bowie’s visual legacy definitely takes up a large piece. But, what does a David Bowie song look like? Valentina D’Efilippo and Miriam Quick answer this question in their remarkable project.

Outfit by Kansai Yamamoto                                Photo by Masayoshi Sukita 1973
Aladdin Sane Cover

OddityViz – a data tribute to David Bowie is a visualization project that gives ‘form to what we hear, imagine and feel while listening to’ Bowie’s hit number Space Oddity. The project which is a combination of ten engraved records, large-scale prints and projections is deconstructed from data extracted from the song – narrative, texture, rhythm, melody, harmony, lyrics, structure, lyrics, trip and emotion. The inquiries that went into the making of each of these records are even more interesting.

When making this, the ninth disc in the Oddityviz series, we asked ourselves: how can we tell the story of Major Tom so it could be understood by an alien?

The project took inspiration from a variety of references from popular culture, while the colour palette naturally recalls the darkness of space (black) and the stars (white). One can also see a reference to the Voyager Golden Records in the engraved dataviz format.

The final disc of the series illustrates the central themes of the song: the destruction of its main character, the bittersweet nature of triumph, the smallness of humanity in a vast, extended universe.

In her article on Muzli, D’Efilippo breaks down the process of creating this piece, comparing the ‘system’ of data visualization to music – one that is largely subjective and that which becomes more ‘meaningful and legible’ as we learn how to read it.

In my opinion, dataviz is more than a tool to render numbers, it’s a way to make sense of any experience and communicate the underpinning stories.

Read the full article here.

Evolution of the Data Artist

Defining Data Art is tricky. And for good reason. The mediascape that breathes around us is a terrain that shifts, distorts and transforms before it can be drawn. In such a space, defining can only be limiting. Jacoba Urist, in his comprehensive article in The Atlantic in 2015 explored the multifarious ways of the Data Artist.

Art is as much a product of the technologies available to artists as it is of the sociopolitical time it was made in, and the current world is no exception. A growing community of “data artists” is creating conceptual works using information collected by mobile apps, GPS trackers, scientists, and more.

                                                      Liberté (1963) – Joaquim Rodrigo 

In a series called Moodjam, (Laurie) Frick took thousands of Italian laminate countertop samples from a recycling center and created a series of canvases and billboard-sized murals based on her temperament … Frick is adamant that her work is about more than simply visualizing information—that it serves as a metaphor for human experience, and thus belongs firmly in the art world.

As Urist deftly puts it – working with (this) data isn’t just a matter of reducing human beings to numbers, but also of achieving greater awareness of complex matters in a modern world. Fast forward to two years later, Cynthia Andrews speaks about the role of Context in Data Art.

If you look at neural networks created by scientists with a creative eye you might see it as art. If you take it out of context, it could be a subway map or a series of rivers. It could be anything. It’s the non-creative context in which things are placed that makes people think they aren’t be considered art.

Andrews expands on a specific genre of Data Art that Urist mentions –

Artists influenced by self-tracking.

‘Waiting for Earthquakes’ by Moon Ribas. She has a sensor embedded into her skin that, using seismic data, vibrates every time there is an earthquake in the world, from anywhere, any magnitude. ‘Waiting for Earthquakes’ is a performance piece in which she literally just stands on stage and waits for an earthquake to happen and then interprets the feeling that she gets into movement. I don’t know if she considers it data art, but I do.

And then, there are artists like Shelita Burke, a pop musician who decided to use Blockchain and Music Metadata to not only get paid on time – but to organize a centralized system for distributing royalties across the production spectrum to the producers and writers involved.

Burke thinks it also has something to do with her use of data to her advantage, like when she determined  that 90 days was the perfect time to release new music in order to keep fans engaged.

“I really believe that every artist needs to understand data” Burke says.

Internet Feudalism v/s Net Neutrality – Who wins?

A few weeks ago, the FCC under the chairmanship of Ajit Pai voted to repeal net neutrality, a topic that soared in Google’s search trends this past December. The interest in the subject when ranked by states (sub-regions) is also quite unexpected with Nebraska at the top, since then becoming the first red state to institute pro net neutrality legislation.

While much is being spoken on the subject including the recent legal resistance from several advocacy groups, the internet association and corporations like Amazon, Google and Netflix, the debate within the wider media continues to remain largely polarized, without taking into account the nuances and hidden realities of the current power structures in place within the world wide web.

Lana Polansky, in her article dives into ‘the emptiness of the myth of the internet as some great equalizer’ and what these feudalistic dynamics mean for independent artists, creators and small businesses even with the existing open internet.

large sections of the internet have been carved out and wholly controlled by major corporations and crowdsourcing and marketplace platforms. The virtual land is farmed for content, from which platform holders skim off profit in exchange for use of the platform.

It has always been difficult for people outside the more privileged classes to hack it as artists and intellectuals, but the break with tradition that the internet was originally believed to represent has now given way to a form of virtual feudalism.

Read the full article here.

The Trouble With Election Maps

The election map in the New York Times was the subject of plenty of conversations in the data visualization and cartography world yesterday. As much as we here at CDA love a good conversation about visual representation (and apparently, we like to do it in rhyme), this map did raise a lot of questions and concerns. In a post for CityLab, Andrew Small writes: “America needs a voting map that actually looks like America.”

Small continues:

But as people tee up to argue and theorize about what the electoral map means for the country, I’m reminded of a recent point of wisdom my colleague Laura Bliss made recently—maps aren’t facts, they’re starting points.

Read Small’s full post for his thoughts on where we can start.