ethics – Data Therapy

Aligning Your Data Methods and Your Mission

This blog post is based on a keynote I gave recently at the 2019 SSIR Data on Purpose event.

Are you optimistic or pessimistic about data as a tool for good in the world? Over the last few years I’ve seen the shift in answers to this questions. People used to answer “optimistic”, but now most people indicate some mix of emotions. You’ve probably seen the Gartner hype cycle with its suggestion that a technology receives inflated expectations and then is overused to the point of disillusionment. I’d argue that the over-hyping and the disappointment happen at the same time… that time is now for social good organizations that are trying to use data to further their missions.

The response I’ve been crafting focuses on acknowledging the damaging history of the data practices we routinely employ, taking a step back from hype-driven roll-out of data programs, and working to align your data and your mission. The Data Culture Project, my collaboration with Catherine D’Ignazio, works with organizations large and small across the world to help make that happen.

One of the core problems in creating a data culture that aligns with your mission is the history we are fighting against. Data has been a tool for those in power to consolidate that power for centuries. For organizations working in the social good sector, this shouldfeel problematic! If you’re deploying some tool or process, you need to be wary of any pieces that reinforce that history. They can cultivate the opposite of the empowerment, engagement, and ownership goals that are probably at the heart of your mission.

Warning: In this post I’m going to depress you by reviewing some of that history. Don’t worry, I’ll close with some inspirations so it isn’t all doom and gloom. Make sure not to stop reading in the middle, otherwise you might walk away feeling pretty bad about the world!

A Depressing History of Data

I won’t surprise anyone by talking about human history as full of stories of those in power seeking to oppress others. However, I want to highlight a few of those instances that were data-driven. Going back as far as the ancient Egyptians, we can find evidence that they tabulated and tracked populations to determine how much labor they would use to construct their giant monuments to themselves (read more at the UK Office for National Statistics). They created census data to drive massive construction projects in their own likeness.

lots of egyptian laborers pulling a large sculpture — Source – A Popular Account of the Ancient Egyptians, Sir John Gardner Wilkinson (1854)

Fast forward to Britain in the 1700s and you find the horrors of the slave trade via cross-atlantic shipping; all cataloged in stunning details by the shopping industry and the strict regulations they were under. Many of these massive data records are still available today. This economic data numerically catalogues the human suffering of tens of millions at the hands of those in power across the western world.

Next consider another of the darkest times in recent history – the Nazi regime. Their massive atrocities were informed and guided by the tabulations of their census, driven by IBM’s custom manufactured counting machines (read more on Wikipedia). This is a history IBM would like us to forget with their new Watson AI initiatives, but Watson was in fact the one that oversaw all this work as CEO at the time.

A few decades later we find another example in South America, recipient of massive investment and development packages from large multi-national UN-style agencies. All this drove numbers that showed enormous economic growth, while in fact huge populations were suffering. As famed author Eduardo Galeano writes – “the more watched and desperate the people, the more the statistics smiled and laughed”.

As for our current times? You can barely throw a stone without hitting another story of a large centralized technology company using some data in morally questionable ways. Amazon pitching facial recognition to ICE here in the US to keep migrants and asylum seekers out (The Verge), Facebook building massive datasets about non-users to improve their invasive advertising targeting (Reuters), China creating a “social credit score” to control and guide citizen norms (Bloomberg) – the dystopia is here. We are all quantized without any meaningful consent in the massive databases of the big tech corporations.

I trust you’re on board now with the idea that data has a dark history full of examples like these ones that I’ve quickly touched on. Want more? Read Algorithms of Oppression, Weapons of Math Destruction, Automating Inequality, or one of the other recent books about this.

A Disempowering Process

I know, that was pretty depressing. Usually I don’t pull in forced labor, slavery, and the Nazi holocaust into one blog post. Sorry. This thing is, this is the past and present that your data process is living within. We can’t ignore it. You have to work hard to make sure you’re not part of it. To fight this history, first we have to understand the patterns that drove it.

All of these examples showcase a problematic pattern of data use that we can characterize in four ways:

Opaque processes– the subjects of the data aren’t given any insight into what happens to the data about them
Extractive collection– the data is pulled from the subjects and their community by those outside of it
High technological complexity– the mechanisms used to analyze the data, digital and non-digital, have a steep learning curve
Control of impact– the people the data is about have no say in the impacts of the decisions being made with the data

From my point of view, these are process problems(read more in my paper on with Catherine D’Ignazio). Ok, in some of the more egregious examples above these wouldn’t be described as “problems”, because clearly their goals were to actively oppress and kill the subjects of the data. However, that isn’t the goal of most data endeavors!

The thing is, even many well-meaning, pro-social data efforts use this problematic process. Consider the history of public health and epidemiology to start. In 1663 you have John Graunt carrying out the first recoded experiments in statical data analysis; the ancestor of epidemiology (learn more on Wikipedia). By recoding information about mortality, he theorized that he could design an early warning system for the bubonic plague ravaging Europe. Definitely working for the social good, but in a position of power with no engagement with the effected populations. Extractive dart collection, a complicated statistical process, and no control of impact for the population in mind.

Or how about the famed maps of John Snow, used to discover the origins of cholera in the early 1800s (learn more on Wikipedia). A noble, impactful, and meaningful example of data usage for sure – literally saving lives. The same process criticisms hold – a person of privilege mapping data about “the poor” to discover something without any role of the people that were the data themselves.

When we quickly read these two stories, they sound like amazing historical examples of using data for good! However, when you examine them more deeply, you find the same four criticisms weighed above. Their data methods didn’t match their mission.

Some Inspirations

Knowing this history, how do you make sure you’re not doomed to repeat it? So how do you avoid these pitfalls? You build a data culture within your organization that can do better. You empower staff up and down your org chart to identify problems data can help solve and them support them solving the those problems. You open up your process, you bring people together, you help them make decisions with data. You don’t need data scientists to to this, you need a data culture. This is what our Data Culture Project is all about. Here are some examples to help explain what I mean.

Two images from the “The Exhibit of American Negroes” exhibit created by W.E.B. Du Bois (source)

A wonderful historical example is the recently re-discovered works of W.E.B. Du Bois. He pulled census data, among other sources, to create a catalog of “the African American” in 1900. He brought the inventive and novel infographics to the world’s fair in Paris to showcase the work that needed to happen to create true freedom in the US post-slavery (work that is still being done today). He worked with African American students at the university to repurpose this census data to tell their story. These graphics are an example of self-determination – highlighting the problems the subjects of the data themselves have chosen. His statistical techniques were detailed, but he invented new ways to communicate them to a larger, less data-literate audience.

A data mural created by the Created by the Collaborate for Healthy Weight Coalition (August 2013)

A general theme in my work is using the arts as an invitation to bring people together around data to tell their own story. My work on data murals, a collaboration with my wife Emily Bhargava, is a prime example of this. We bring a group of people together around some data about them, help them find a story they want to tell, and then collaboratively design a mural to tell it. This puts the ownership of the data analysis and the data story in their hands, flipping the standard process on its head. The subjects of the data are empowering to tell the data story, with invitations to analyze the data that build their capacity and meet them where they are in skills and interests.

Data 4 Black Lives founders Yeshimabeit Milner, Lucas Mason-Brown, and Max Clermont

A more community-focused example comes from the Data 4 Black Lives organization(D4BL). The brutal legacy of slavery in the US permeates our culture, so it should be no surprise that it continues to poison the datasets we use as well. D4BL is working to highlight these problems, bring together organizers, data scientists, and students, and also influence policy to put data in service of the needs of black lives in the US. This is a traditionally marginalized community empowering themselves with the language of those in power (data) and trying to build community in service of their own goals.

hearts and a woman seeming to be in pain, half the hearts are filled in, half are empty — 1st place winner Danford Marco and his Khanga design

For a non-US example, we can look to the work done by the Tanzania Bhora Initiative and Faru Arts and Sports Development Organizatio as part of the Data Zetu Intiaitive in Tanzania. They ran a competition for designers to create khanga cloth patterns based on data (the khanga is a traditional cotton clothoften adorned with sayings or shout-outs). The project built the capacity of designers to speak data, and ended with a fashion show showcasing the winning designs (read more in their blog post). The first place winner (Danford Marco) created a design to reflect that 1 out of every 2 married women have faced some kind of abuse from their husband. A staggering statistic, and a stunning design to bring attention to the problem. This kind of creative approach to building data capacity is an example of a very different process, one that is inclusive, builds capacity, and gives ownership of the data to the people it is about.

Match your Mission and Methods

an arrow connecting data to a building with a heart — Align your data methods and your mission

I’m hoping by now that I’ve convinced you that you need to think harder about the data methods you use in order to avoid re-creating the terrible historical practice associated with data. I’m focused on organizations that work for the social good, but this argument holds true for anyone using data. The inspirational examples I highlight all paint a path forward that lets us match our mission and our methods. Which path will you follow?

data literacy, ethics

The algorithms aren’t biased, we are

Excited about using AI to improve your organization’s operations? Curious about the promise of insights and predictions from computer models? I want to warn you about bias and how it can appear in those types of projects, share some illustrative examples, and translate the latest academic research on “algorithmic bias”.

First off – language matters. What we call things shapes our understanding of them. That’s why I try to avoid the hype-driven term “artificial intelligence”. Most projects called that are more usefully described as “machine learning”. Machine learning can be described as the process of training a computer to make decisions that you want help making. This post describes why you need to worry about the data in your machine learning problem.

This matters in a lot of ways. “Algorithmic bias” is showing up all over the press right now. What does that term mean? Algorithms are doling our discriminatory sentence recommendations for judges to use. Algorithms are baking in gender stereotypes to translation services. Algorithms are pushing viewers towards extremist videos on YouTube. Most folks I know agree this is not the world we want. Let’s dig into why that is happening, and put the blame where it should be.

Your machine is learning, but who is teaching it?

Physics is hard for me. Even worse – i don’t think I’ll ever be good at physics. I attribute a lot of this to a poor high school physics teacher, who was condescending to me and the other students. On the other hand, while I’m not great at complicated math, I like trying to learn it better. I trace this continued enthusiasm to my junior high school math teacher, who introduced us to the topic with excitement and playfulness (including donut rewards for solving bonus problems!).

My point in sharing this story? Teachers matter. This is even more true in machine learning – machines don’t bring prior experience, contextual beliefs, and all the other things that make it important to meet human learners where they are and provide many paths into content. Machines only learn from only what you show them.

So in machine learning, the questions that matter are “what is the textbook” and “who is the teacher”. The textbook in machine learning is the “training data” that you show to your software to teach it how to make decisions. This usually is some data you’ve examined and labeled with the answer you want. Often it is data you’ve gathered from lots of other sources that did that work already (we often call this a “corpus”). If you’re trying to predict how likely someone receiving a micro-loan is to repay it, then you might pick training data that includes previous payment histories of current loan recipients.

The second part is about who the teacher is. The teacher decides what questions to ask, and tells learners what matters. In machine learning, the teacher is responsible for “feature selection” – deciding what pieces of the data the machine is allowed to use to make its decisions. Sometimes this feature selection is done for you by what is and isn’t included in the training sets you have. More often you use some statistics to have the computer pick the features most likely to be useful. Returning to our micro-loan example: some candidate features could be loan duration, total amount, whether the recipient has a cellphone, marital status, or their race.

These two questions – training data and training features – are central to any machine learning project.

Algorithms are mirrors

Let’s return to this question of language with this in mind.. perhaps a more useful term for “machine learning” would be “machine teaching”. This would put the responsibility where it lies, on the teacher. If you’re doing “machine learning”, you’re most interested in what it is learning to do. With “machine teaching”, you’re most interested in what you are teaching a machine to do. That’s a subtle difference in leanguage, but a big difference in understanding.

Putting the responsibility on the teacher helps us realize how tricky this process is. Remember this list of biases examples I started with? That sentencing algorithm is discriminatory because it was taught with sentencing data for the US court system, which data shows is vey forgiving to everyone except black men. That translation algorithm that bakes in gender stereotypes was probably taught with data from the news or literature, which we known bakes in our-of-date gender roles and norms (ie. Doctors are “he”, while nurses are “she”). That algorithm that surfaces fake stories on your feed is taught to share what lots of other people share, irrespective of accuracy.

All that data is about us.

Those algorithms aren’t biased, we are! Algorithms are mirrors.

They reflect the biases in our questions and our data. These biases get baked into machine learning pejects in both feature selection and training data. This is on us, not the computers.

Corrective lenses

So how do we detect and correct this? Teachers feel a responsibility for, and pride in, their students’ learning. Developers of machine learning models should feel a similar responsibility, and perhaps should be allowed to feel a similar pride.

I’m heartened by examples like Microsoft’s efforts to undo gender bias in publicly available language models (trying to solve the “doctors are men” problem). I love my colleague Joy Buolamwini’s efforts to reframe this as a question of “justice” in the social and technical intervention she calls the “Algorithmic Justice League” (video). ProPublica’s investigative reporting is holding companies accountable for their discriminatory sentencing predictions. The amazing Zeynep Tufekci is leading the way in speaking and writing about the danger this poses to society at large. Cathy O’Neil’s Weapons of Math Destruction documents the myriad of implications for this, raising a warning flag for society at large. Fields like law are debating the implications of algorithm-driven decision making in public policy settings. City ordinances are started to tackle the question of how to legislate against some of the effects I’ve described.

These efforts can hopefully serve as “corrective lenses” for these algorithmic mirrors – addressing the troubling aspects we see in our own reflections. The key here is to remember that it is up to us to do something about this. Determining a decision with an algorithm doesn’t automatically make it reliable and trustworthy; just like quantifying something with data doesn’t automatically make it true. We need to look at our own reflections in these algorithmic mirrors and make sure we see the future we want to see.

ethics

Creating Ethical Algorithms – Data on Purpose Live Blog

This is a live-blog from the Stanford Data on Purpose / Do Good Data “From Possibilities to Responsibilities” event. This is a summary of what the speakers at the talked about, captured by Rahul Bhargava and Catherine D’Ignazio. Any omissions or errors are likely my fault.

Human-Centered Data Science for Good: Creating Ethical Algorithms

Zara Rahman works at both Data & Society and the Engine Room, where she helps co-ordinate the Responsible Data Forum series of events. Jake Porway founded and runs DataKind.

Jake notes this is the buzzkill session about algorithms. He wants us all to walk away being able to critically assess algorithms.

How do Algorithms Touch our Lives?

They invite the audience to sketch out their interactions with digital technologies over the last 24 hours on a piece of paper. Stick figures and word totally ok. One participant drew a clock, noting happy and sad moments with little faces. Uber and AirBnb got happy faces next to them. Trying to connect to the internet in the venue got a sad face. Here’s my drawing.

Next they ask where people were influenced by algorithms. One participant shares the flood warning we all received on our phones. Another mentioned a bot in their Slack channel that queued up a task. Someone else mentions how news that happened yesterday filtered down to him; for instance Hans Rosling’s death made it to him via social channels much more quickly than via technology channels. Someone else mentioned how their heating had turned on automatically based on the temperature.

What is an Algorithm?

Jake shares that the wikipedia-esque definition is pretty boring. “A set of rules that precisely deinfes a sequence of operations”. These examples we just heard demonstrate the reality of this. These are automated and do things on their own, like Netflix’s recommendation algorithm. The goal is to break down how these operate, and figure out how to intervene in what drives these thinking machines. Zara reminds us that even if you see the source code, that doesn’t help really understand it. We usually just see the output.

Algorithms have some kind of goal they are trying to get to. It takes actions to get there. For Netflix, the algorithm is trying to get you to watch more movies; while the actions are about showing you movies you are likely to want to watch. It tries to show you movies you might like; there is no incentive to show you a movie that might challenge you.

Algorithms use data to inform their decisions. In Netflix, the data input is what you have watched before, and what other people have been watching. There is also a feedback loop, based on how it is doing. It needs some way to figure out it is doing a good thing – did you click the movie, how much of it did you watch, how many star did you give it. We can speculate about what those measurements are, but we have no way of knowing their metrics.

A participant asks about how Netflix is probably also nudging her towards content they have produced, since that is cheaper for them. The underlying business model can drive these algorithms. Zara responds that this idea that the algorithm operates “for your benefit” is very subjective. Jake notes that we can be very critical about their goal state.

Another participant notes that there are civic benefits; in how Facebook can influence how many people are voting.

The definition is tricky, notes someone else, because anything that runs automatically could be called an algorithm. Jake and Zara are focused in on data-driven algorithms. They use information about you and learning to correct themselves. The purest definition and how the word is used in media are very different. Data science, machine learning, artificial intelligence – these are all squishy terms that are evolving.

Critiquing Algorithms

They suggest looking at Twitter’s “Who to follow” feature. Participants break into small groups for 10 minutes to ask questions about this algorithm. Here are the questions and some responses that groups shared after chatting:

What is the algorithm trying to get you to do?
- They want to grow their user base, and then shifted to growing ad dollars
- Showing global coverage, to show they are the network to be in
- People name some unintended consequences like political polarization
What activities does it use to do that?
What data drives these decisions?
- Can you pay for these positions? There could be an agreement based on what you are looking at and what Twitter recommends
What data does it use to evaluate if it is successful?
- It can track your hovers, clicks, etc. both on the recommendation and adds later on
- If you don’t click to follow somewhere that could be just as much signal
- They might track the life of your relationship with this person (who you follow later because you followed their recommendation, etc)
Who has the power to influence these answers?

A participant notes that there were lots of secondary outcomes, which affected other people’s products based on their data. Folks note that the API opens up possibilities for democratic use and use for social good. Others note that Twitter data is highly expensive and not accessible to non-profits. Jake notes problems with doing research with Twitter data obtained through strange and mutant methods. Another participant notes they talked about discovering books to read and other things via Twitter. These reinforced their world views. Zara notes that these algorithms reinforce the voices that we hear (by gender, etc). Jake notes that Filter Bubble argument, that these algorithms reinforce our views. Most of the features they bake in are positive ones, not negative.

But who has the power the change these things? Not just on twitter, but health-care recommendations, Google, etc. One participant notes that in human interactions they are honest and open, but online he lies constantly. He doesn’t trust the medium, so he feeds it garbage on purpose. This matches his experiences in impoverished communities, where destruction is a key/only power. Someone else notes that the user can take action.

A participant asks what the legal or ethical standards should be. Someone responds that in non-profits the regulation comes from self-regulation and collective pressure. Zara notes that Twitter is worth nothing without it’s users.

Conclusion

Jake notes that we didn’t talk about it directly, but the ethical issues come up in relation to all these questions. These systems aren’t neutral.

big data, data literacy, ethics

Practicing Data Science Responsibly

I recently gave a short talk at a Data Science event put on by Deloitte here in Boston. Here’s a short write up of my talk.

Data science and big data driven decisions are already baked into business culture across many fields. The technology and applications are far ahead of our reflections about intent, appropriateness, and responsibility. I want to focus on that word here, which I steal from my friends in the humanitarian field. What are our responsibilities when it comes to practicing data science? Here are a few examples of why this matters, and my recommendations for what to do about it.

Practicing Data Science Responsibly from rahulbot

People Think Algorithms are Neutral

I’d be surprised if you hadn’t heard about the flare-up about Facebook’s trending news feed recently. After breaking on Gizmodo if has been covered widely. I don’t want to debate the question of whether this is a “responsible” example or not. I do want to focus on what it reveals about the public’s perception of data science and technology. People got upset, because they assumed it was produced by a neutral algorithm, and this person that spoke with Gizmodo said it was biased (against conservative news outlets). The general public thinks algorithms are neutral, and this is a big problem.

Algorithms are artifacts of the cultural and social contexts of their creators and the world in which they operate. Using geographic data about population in the Boston area? Good luck separating that from the long history of redlining that created a racially segregated distribution of ownership. To be responsible we have to acknowledge and own that fact. Algorithms and data are not neutral third parties that operate outside of our world’s built-in assumptions and history.

Some Troubling Examples

Lets flesh this out a bit more with some examples. First I look to Joy Boulamwini, a student colleague of mine in the Civic Media group at the MIT Media Lab. Joy is starting to write about “InCoding” – documenting the history of biases baked into the technologies around us, and proposing interventions to remedy them. One example is facial recognition software, which has consistently been trained on white male faces; to the point where she has to literally done a white-face mask to have the software recognize her. This just the tip of the iceberg in computer science, which has a long history of leaving out entire potential populations of users.

Another example is a classic one from Latanya Sweeney at Harvard. In 2013 She discovered a racial bias trained into the operation Google’s AdWords platform. When she searched for names that are more commonly given to African Americans (liked her own), the system popped up ads asking if the user wanted to do background checks or look for criminal records. This is an example of the algorithm reflecting built-in biases of the population using it, who believed that these names were more likely to be associated with criminal activity.

My third example comes from an open data release by the New York City taxi authority. They anonymized and then released a huge set of data about cab rides in the city. Some enterprising researchers realized that they had done a poor job of anonymizing the taxi medallion ids, and were able to de-anonymize the dataset. From there, Anthony Tockar was able to find strikingly juicy personal details about riders and their destinations.

A Pattern of Responsibility

Taking a step back form these three examples I see a useful pattern for thinking about what it means to practice data science with responsibility. You need to be responsible in your data creation, data impacts, and data use. I’ll explain each of those ideas.

Being responsible in your data collection means acknowledging the assumptions and biases baked into your data and your analysis. Too often these get thrown away while assessing the comparative performance between various models trained by a data scientist. Some examples where this has failed? Joy’s InCoding example is one of course, as is the classic Facebook “social contagion” study. A more troubling one is the poor methodology used by US NSA’s SkyNet program.

Being responsible in your data impacts means thinking about how your work will operate in the social context of its publication and use. Will the models you trained come with a disclaimer identifying the populations you weren’t able to get data from? What are secondary impacts that you can mitigate against now, before they come back to bite you? The discriminatory behavior of the Google AdWords results I mentioned earlier is one example. Another is the dynamic pricing used by the Princeton Review disproportionately effecting Asian Americans. A third are the racially correlated trends revealed in where Amazon offers same-day delivery (particularly in Boston).

Being responsible in your data use means thinking about how others could capture and use your data for their purposes, perhaps out of line with your goals and comfort zone. The de-anonymization of NYC taxi records I mentioned already is one example of this. Another is the recent harvesting and release of OKCupid dating profiles by researchers who considered it “public” data.

Leadership and Guidelines

The problem here is that we have little leadership and few guidelines for how to address these issues in responsible ways. I have yet to find an handbook for a field that scaffolds how to think about these concerns. As I’ve said, the technology is far ahead of our reflections on it together. However, that doesn’t mean that they aren’t smart people thinking about this.

In 2014 the White House brought together a team to create their report on Big Data: Seizing Opportunities, Preserving Values. The title itself reveals their acknowledgement of the threat some of these approaches have for the public good. Their recommendations include a number of things:

extending the consumer bill of rights
passing stronger data breach legislation
protecting student centered data
identifying discrimination
revising the Electronic Communications Privacy Act

Legislations isn’t strong in this area yet (at least here in the US), but be aware that it is coming down the pipe. Your organization needs to be pro-active here, not reactive.

Just two weeks ago, the Council on Big Data, Ethics and Society released their “Perspectives” report. This amazing group of individuals was brought together to create this report by a federal NSF grant. Their recommendations span policy, pedagogy, network building, and area for future work. The include things like:

new ethics review standards
data-aware grant making
case studies & curricula
spaces to talk about this
standards for data-sharing

These two reports are great reading to prime yourself on the latest high-level thinking coming out of more official US bodies.

So What Should We Do?

I’d synthesize all this into four recommendations for a business audience.

Define and maintain our organization’s values. Data science work shouldn’t operate in a vacuum. Your organizational goals, ethics, and values should apply to that work as well. Go back to your shared principles to decide what “responsible” data science means for you.

Do algorithmic QA (quality and assurance). In software development, the QA team is separate from the developers, and can often translate between the languages of technical development and customer needs. This model can server data science work well. Algorithmic QA can discover some of the pitfalls the creators of models might not.

Set up internal and and external review boards. It can be incredibly useful to have a central place where decisions are made about what data science work is responsible and what isn’t for your organization. We discussed models for this at a recent Stanford event I was part of.

Innovate with others in your field to create norms. This stuff is very new, and we are all trying to figure it out together. Create spaces to meet and discuss your approaches to this with others in your industry. Innovate together to stay ahead of regulation and legislation.

These four recommendations capture the fundamentals of how I think businesses need to be responding to the push to do data science in responsible ways.

This post is cross-posted to the civic.mit.edu website.

big data, ethics

Ethical Data Review Procesess Workshop at Stanford

The Digital Civil Society Lab at Stanford recently hosted a small gathering of people to dig into emerging processes for ethical data review. This post is a write up of the publicly shareable discussions there.

Introduction

Lucy Berholz opened the day by talking about “digital civil society” as an independent space for civil society in the digital sphere. She is specifically concerned with how we govern the digital sphere in line with the a background of democracy theory. We need to use, manage, govern in ways that are expansive and supportive for independant civil society. This requires new governance and review structures for digital data.

This prompted the question of what is “something like an IRB and not an IRB”? The folks in the room bring together corporate, community, and university examples. These encompass ethical codes and the processes for judging adherence to them. With this in mind, in the digital age, do non-profits need to change? What are the key structures and governance for how they can manage private resources for public good?

Short Talks

Lucy introduced a number of people to give short talks about their projects in this space.

Lasanna Magassa (Diverse Voices Project at UW)

Lasanna introduced us all to the Diverse Voices Project, an “An exploratory method for including diverse voices in policy development for emerging technologies”. His motivations lie in the fact that tech policy is generally driven by mainstream interests, and that policy makers are reactive.

They plan and convene “Diverse Voices Panels”, full of people whole live an experience, institutions that support them, people somehow connected to them. In a panel on disability this could be people who live it and are disabled, law & medical professionals, and family members. These panels produce whitepapers that document and then make recommendations. They’ve tackled everything from ethics and big data, to extreme poverty, to driverless cars. They focus on what technology impacts can be for diverse audiences. One challenge they face is finding and compensating panel experts. Another is wondering how to prep a dense, technical document for the community to read.

Lasanna talks about knowledge generation being the key driver, building awareness of diversity and the impacts of technologies on various (typically overlooked) subpopulations.

Eric Gordon (Engagement Lab at Emerson College)

Eric (via Skype) walked us through the ongoing development of the Engagement Lab’s Community IRB project. The goal they started with was to figure out what a Community IRB is (public health examples exist). It turned out they ran into a bigger problem – transforming relationships between academia and community in the context of digital data. There is more and more pressure to use data in more ways.

He tells us that in Boston area, those who represent poorer folks in the city are asked for access to those populations all the time. They talked to over 20 organizations about the issues they face in these partnerships, focusing on investigating the need for a new model for the relationships. One key outcome was that it turns out nobody knows what an IRB is; and the broader language use to talk about them is also problematic (“research”, “data”).

They ran into a few common issues to highlight. Firstly, there weren’t clear principles for assuring value for those that give-up their data. In addition, the clarity of the research ask was often weak. There was a all-to-common lack of follow-through, and the semester-driven calendar is a huge point of conflict. An underlying point was that organizations have all this data, but the outside researcher is the expert that is empowered to analyze it. This creates anxiety in the community organizations.

They talked through IRBs, MOUs, and other models. Turns out people wanted to facilitate between organizations and researchers, so in the end what they need is not a document, but a technique for maintaining relationships. Something like a platform to match research and community needs.

Molly Jackman & Lauri Kanerva (Facebook)

Molly and Lauri work on policy and internal research management at Facebook. They shared a draft of the internal research review process used at Facebook, but asked it not be shared publicly because it is still under revision. They covered how they do privacy trainings, research proposals, reviews, and approvals for internal and externally collaborative research.

Nicolas de Corders (Orange Telekom)

Nicolas shared the process behind their Data for Development projects, like their Ivory Coast and Senegal cellphone data challenges. The process was highly collaborative with the local telecommunications ministries of each country. Those conversations produced approvals, and key themes and questions to work on within the country. This required a lot of education of various ministries about what could be done with the cellphone call metadata information.

For the second challenge, Orange set up internal and external review panels to handle the submissions. The internal review panels included Orange managers not related to the project. The external review panel tried to be a balanced set of people. They built a shared set of criteria by reviewing submissions from the first project in the Ivory Coast.

Nicolas talks about these two projects as one-offs, and scaling being a large problem. In addition, getting the the review panels to come up with shared agreement on ethics was (not surprisingly) difficult.

Breakouts

After some lunch and collaborative brainstorming about the inspirations in the short talks, we broke out into smaller groups to have more free form discussions about topics we were excited about. These included:

an international ethical data review service
the idea of minimum viable data
how to build capacity in small NGOs to do this
a people’s review board
how bioethics debates can be a resource

I facilitated the conversation about building small NGO capacity.

Building Small NGO Capacity for Ethical Data Review

Six of us were particularly interested in how to help small NGOs learn how to ask these ethics questions about data. Resources exist out there, but not well written enough for people in this audience to consume. The privacy field especially has a lot of practice, but only some of the approaches there are transferrable. The language around privacy is all too hard to understand for “regular people”. However, their approach to “data minimization” might have some utility.

We talked about how to help people avoid extractive data collection, and the fact that it is disempowering. The non-profit folks in the group reminded us all that you have to think about the funder’s role in the evidence they are asking for, an how they help frame questions.

Someone mentioned that law can be the easiest part of this, because it is so well-defined (for good or bad). We have well established laws on the fundamental privacy right of individuals in many countries. I proposed participatory activities to learn these things, like perhaps a group activity to try and re-identify “anonymized” data collected from the group. Another participant mentioned DJ Patel’s approach to building a data culture.

Our key points to share back with the larger group were that:

privacy has inspirations, but it’s not enough
communications formats are critical (language, etc); hands-on, concrete, actionable stuff is best
you have to build this stuff into the culture of the org