In defense of OTHER

I was trying to get the registration stickers for my car. I should’ve gotten them, but I hadn’t. Instead, I’d gotten three different letters from the DMV informing me that my registration was paid but not complete. The automated system wouldn’t send the stickers. No one knew why; certainly not me, since I had paid the registration far in advance and gotten the required smog test, and not the woman at the DMV who answered after I sacrificed three hours on hold in the traditional supplication.

“I see that you paid, and I see your vehicle. There’s just a code associated with your vehicle,” was what she told me. I had been working on a puzzle during my hours on hold. This is a waiting activity I recommend. I couldn’t trust the automated call-back queue after they hadn’t actually called me back. So instead I had been hanging onto the line and reducing the waiting music to a tolerable level and finishing puzzles.

“Can you help me understand what that means?” I asked. We were two people running scripts at each other. Hers from whatever gods helm the DMV and mine from qualitative interviewing and time spent embedded on engineering teams.

After a long pause, the woman on the phone said: “I have no idea." She sounded sincere and human, which is to say, she sounded pissed. Script broken, contact established. “They don’t give us anything but the code. All I can do is look up what the code stands for. This one stands for OTHER.”

---

OTHER is a category paradox, a response to say response not covered.

Confession: as a researcher I have written many OTHER categories into existence at the point of data collection. I know, I know. I know it's unsatisfying. Usually these are surveys. Sometimes they are my own labelling of a scenario, or scoring of a performance. Sometimes they are bins into which I shunt a hundred thousand or so datapoints. Response not covered. Interesting but unfortunately not the point of this study. Someone decided not to instrument whatever this was so I can’t make a prediction of it. I don't trust it. I was surprised.

I know also that I choose to go into this world of measurement trade-offs. Research is always a sandbox, always a construction of a limited field. There are a lot of possible things you could store in a spice cupboard but it’s understandable your labels would say pepper and not dragons. Yet there are also good reasons to open a paradox, even a tiny one, even in your sandbox. Sometimes you have limited fields in the free version of the survey software your lab lets you use. Sometimes you only have a few labels that you feel confident spelling, defining, or throwing out to your participants, and you want to give them the chance to be generative participants, not just reactive participants. Sometimes you don’t trust the people you’re working with to protect the complexity of your participants and you write OTHER for things you already know, or live (and you hope that it doesn’t hurt people who know, or live, like you).

We--data people, researchers, some random engineer at the DMV doing her best--create standardized responses because we want to reify patterns. This requires cutting things up and picking dividing lines, even when we know real things might bleed over each other in the margins. We accept a lot of penalties when we define measurements: the flattening of vast spectra of experience into homogenous ranks and order numbers, the “best-of” choice that we know every reader has to make, the extraordinarily profound limits of our own imaginations. Still it’s a tool. We live for patterns. We live because of best-of choices, sometimes.

And then sometimes we use OTHER. OTHER is a pattern breaker. OTHER is an admittance of the missing. Writing OTHER as a response option, every quantitative researcher knows, takes away some power from your analysis and shifts it into the unknown investment that is personalized accuracy. Stakeholders don’t like this. Stakeholders don’t often hire you for this.

But OTHER can be beautiful, a validation, a pilot test in miniature, a rummage bin for people's unwrangled self-words, their non-standardized representations.

---

The woman at the DMV gave me the phone number of the Meta DMV, DMV Plus, the place where codes go to die. My problem had become some kind of higher order problem, and I needed a translator. I opened up another puzzle.

Meta DMV had the kind of hours that speak to organizations under meta stressors. You could only reach them during a lunch hour window on a particular weekday. There was no callback waiting queue.

I called and called and called. I logged in and out of the DMV website. The OTHER code was not visible on my customer UI. It was invisible, backend, logged somewhere. I had already paid my registration so I couldn’t pay it again (I was about ready to). I didn't want to drive my car with an expired (??) registration so OTHER cost me experience as well as time.

---

OTHER can be insulting. OTHER can be painful. I have no desire to pretend it’s not, least of all when otherization in the sociological sense is itself a decision with cascading permutations. Those included and excluded from “default” definitional measurement have always known this. This exclusion shows up in our data science, and in our standard methodologies, and is all the worse for being invisible to so many who create those methodologies.

I have faced down a lot of surveys where OTHER is the only label that fits my response and it does not feel like it will ever help: for example, at many “women in tech” events that might discuss sexism without ever imagining the ways in which gender-expansive people experience different weaponizations. The push for data-driven rarely extends to being measurement-driven. Being data-driven is usually about the point of analysis, not the point of collection. This is why they ask so much about coding languages and so little about causal inference in most data-related job interviews.

I’ve lived through a thousand mundane moments of grief about losing rich measurement working in this field. Data cleaning courses seem to spend a lot of time on dumping, not expanding. The fact that quantitative work sees success as majoritizing patterns means that what we dump in OTHER itself is rarely random and often systematic. Plus feature reduction is efficient and efficiency is, of course, profit. Being called inefficient as an applied researcher can feel like a death knell.

So it’s not a fix, using OTHER. I think it can be like cracking open a window, though. There is always a tension between exploratory research and confirmatory research. There is always a struggle to own the definitions and not have them applied to you. It is difficult to attain the power and the agency to create labels. Sometimes what you can do is sneak in an OTHER. Sometimes it’s breathing room against all of the frustration, a tiny little moment where you say: hey. YOU tell me.

---

I finished two puzzles in the course of never getting anyone at the Meta DMV to answer my calls. Several weeks later, I got a very tiny reimbursement check. Mystery thickened! Then I got a letter with my sticker that said: your process is complete.

(Ok, I never found out what the code meant. But here is my best guess: I had moved, but I had paid a registration fee with my car still listed to an old address without realizing it, maybe even while my address change was still in process. My registration had therefore been overcharged by the value of approximately three dollars and fifteen cents. This shunted my registration away from normal DMV processing, and into some baroque and time-dilated reality of reimbursement. They probably couldn’t send the sticker before they’d processed the reimbursement. Edge case, conditionalities I couldn't see, OTHER. Or maybe it was pandemic vibes hitting the DMV, I don't know. Everything has been an OTHER since March 2020 hasn't it.

I also got a ticket while my car was parked next to my house waiting for its sticker! So going nowhere didn't save me from the OTHER. I explained that I had paid registration: OTHER was not an acceptable supplication to that set of gods, who had codes of their own, the ticket was dismissed but I had to pay some kind of service fee and reduced ticket for existing while coming to their attention. Category? Probably OTHER)

I’m not telling this story to complain about it--it was kinda stupid to experience, but hardly heinous--but because I keep thinking about it. The invisibility of what that OTHER was supposed to stand for. The impossibility of solving a situation once the definitions were rendered unreachable. That woman on the phone who was allowed to read the labels but not see what they meant. Sometimes when I encounter an OTHER category in my data I play that excellent Sesame Street song in my head, one of these things is not like the other.

This trivial DMV story is an anti-example. I thought about writing down OTHER stories where the label worked--qualitative insights from open text survey responses that drove decision making and changed what we worked on next, or ways that I’ve used mixed-methods research to try to co-create data with participants in the first place. But this story just kept sticking in my head. We’re surrounded by OTHER everyday. We’re dealing with outliers by not dealing with them. We’re deciding that our categorizations must work because we have a code for them not-working, and therefore our sorting of things-into-categories is one hundred percent effective.

Despite this some of us become OTHER spotters, likely to volunteer for the labor of reading the text. I guess I'm one of those. In the classic Sesame Street song you have thirty seconds to identify which thing is not like the others and I think that mounting tension is very accurate. But I like OTHER. It can be a terrible solution but not having that strange category feels like it could be worse. Pretending that our responses are exhaustive when they are not. Worshipping at the church of standard limited options. Pre-selecting only for participant reaction, not participant generation. OTHER is dodgy, a standardization-breaker, but also a space for imagination. I like to think that it lets us admit that we have not drawn a boundary around the entire world just because we wrote out five possible answers.

I try not to create invisibilities masked by OTHER. I still like writing those small OTHER options whenever I can. I know, inefficiency, etc. I know that harried quantitative researchers or dismissive data scientists will probably ifelse(x = OTHER, delete) or whatever. But once in a while it must get through. Once in a while someone must read it, and consider the existence of a category they never imagined.

Telling your own stories in career transitions

Sometimes I give guest lectures to grad students about transitioning from academia to careers in industry. I always really enjoy these; I think the analytical, problem-solving, and insight skills of grads are amazing, and widely applicable to a growing number of jobs. At the same time, it can be such a difficult transition to navigate. I often get a ton of questions that center on one general theme: how do we make a case for ourselves when we're trying to change fields and don't feel like we have the same background as everyone else applying?

I get it. It really can be hard. When I was transitioning into industry, I felt lost, and muddled through a whole lot of processes that I didn’t understand. How do you convince a whole different field, especially one that speaks a different jargon from yours, that you’re valuable? How do you “market” yourself while remaining authentic? How do you mine your experience for relevant work, when you haven’t been given access to the type of work you want to transition into?

After you’ve boiled your CV into a resume, browsed the alt-ac tags on medium, and followed a few ten-step-checklists for scouring job ads, it can still be hard to know how to get across the tougher hurdles. But here’s something that really helped me: Talk about your experience and skills with a storytelling framework.

Along with research and stats, I write a lot of fiction. Lately, I’ve been thinking about how much the tools of narrative have helped me figure out how to talk about my professional identity. I think it helped me over numerous career transitions, so here are a few ways I think about it.

You get to be the protagonist.

Unlike in the academic research world, where we are trained to decenter our perspective and speak only of our findings as disembodied facts, you can make your career story about yourself. In fact, you have to. Think of yourself as the protagonist.

I found this a difficult switch. I hid behind passive jargon and labels, instead of describing my experience as a series of actions. I said that I worked on “developmental social cognition,” but not that I had sat every saturday morning in the entranceway of the local aquarium, patiently collecting hundreds of children’s responses to storybook scenarios. I said that I had “generated research articles,” but not that I had learnt to write an empirically-reviewed research paper and gone through the arduous process of incorporating scientific feedback. My career story had no life, and it was hard to understand from the outside, because I was asking people to infer so much about specialized, academic experiences.

But you really can treat yourself like the protagonist. Once you do that, other people can, too. Protagonists are accessible: we know them by their actions. Look at whether you've described your positions as a journey, where you made choices. Were these roles something that happened to you? Or can you find an outcome that you caused with your decision-making? As I began to uncover the protagonist perspective in how I talked about my career, everything I said changed. Things I hadn't thought about as useful skills before became key elements to the story. It wasn't just about dryly listing the outcomes of how many experiments I'd done, but why I'd chosen one experiment and method over another. I was a mentor to over twenty research assistants, and I spent extra time working on cross-cultural methodologies. It was decision-making and problem-solving.

Turn obstacles into quests.

Often when I help students revise cover letters or resumes, I notice they spend time justifying or explaining gaps in their experience. This can look like saying, “I know one of the job requirements is machine learning, and I only ever worked in a psychology lab….” or “even though I don’t have a degree in computer science…” I understand the impulse to explain these things! When we feel different from other people on the path toward a certain career, we want to acknowledge and justify our different skills. But this language can accidentally come off as overly defensive.

Thinking in terms of your overall story can also help with this. In stories, the protagonist will always encounter obstacles. In fact, it’s pretty boring if everything goes perfectly. One of the tricks I use to try to overcome my own worry about this is to imagine that these are quests, not limitations. The important thing for a protagonist is getting through the quests, not avoiding them altogether. Maybe the quest sets them on a new, unexpected path.

This is how I try to think about telling my story in a new field, too. I went to an undergrad that didn’t have computer science courses available to most students. I navigated into tech from the outside, with more of my years spent in social science. But my story changed when I learned to see that teaching myself to code on my own, outside of a classroom, was a valuable and unique experience instead of something that I had to dismiss or justify. The fact that I wasn't steeped in engineering culture lent me a fresh perspective in appreciating the human problems we were working on in applied research. It all became part of my story.

Make your good ending seem inevitable.

There’s a maxim about writing that says something like, “the end is always in the beginning.” This idea has always hit home for me--a really good story will echo pieces of the beginning, and even across unexpected twists and turns, we feel satisfied when it concludes in a place that seems right. Part of what you’re doing when you’re applying to a job (or any other situation where you need to convince someone of your skills) is weaving together a compelling story that makes you the fit.

This is the first thing I look for when I give feedback on student essays and cover letters: could nearly every sentence that lists out facts about their experience end with something like “...and that’s why I will be a good fit for this job”? Obviously, not everything will connect so directly, but the end should be clear, and it should be about the future. The next step in my journey should be this one.

This is scary for people trying to transition into a new field. It was scary for me. For a long while, I covered this fear by continuing to pile more facts and accomplishments into my career story. I would list bonus UX contracts that I did, or esoteric statistics models I’d figured out. These were all good things, but they weren’t actually about what I wanted. I was afraid to put passion, clarity, and desire into my career story. I constantly got feedback from peers that my cover letters and introductions were simply too dense. The future was getting lost in the past. I had workshopped my materials and identified my skills and thought of myself as the protagonist--but I hadn’t taken the final, crucial step of really believing in the future I was going after.

Finally, I made this switch. After all, we need to know what the protagonist cares about, so that we can care too. I re-drafted my cover letters to start and end with what I wanted: to work on real problems that impacted human beings, to use good ethical measurement to help empower people, and to work on a collaborative, fun team. These wishes and desires, backed up by more carefully-chosen facts, and given unique color by the obstacles I’d faced, became the heart of my career story. And when I let that heart have room, people noticed. They remembered what I’d said I was looking for, and connected me to more opportunities. I got more interviews, and found gigs that lined up with my values.

Whether or not you need to navigate into a new field like I did, I think working on a career story can be profoundly useful. I still come back to these exercises often. And we can have multiple stories. I'm constantly revising mine; another maxim about writing is that you never really end a story, you just choose a stopping place. So, here's one. :)

When we ask for help in learning

Originally posted to medium here

I’ve always been the kind of person who forgets it’s an option to raise your hand and ask a question. I nearly slipped down a grade in a history class despite unimpeachable exams because I hardly opened my mouth during lectures; it just seemed kind of silly to me that there was a participation requirement in a history class. History has, after all, already happened. I was usually a student who just wanted to listen.

Nevertheless, fortune favors the bold…which we say, mostly, about the loud. Which are really the people who are allowed to be loud. After all, education systems are both the explicit systems that we see and the shadow systems underneath, that tricky multidimensional thing that operates with long chains of consequence.

There are many rules for being a student, and many ways that rules operate, and then many possible consequences of breaking those rules, depending on who you are. Research in sociology and other fields continually illuminates how much secret advantage exists in who can negotiate and for what in school. We know that even very young children worry about reactions to disclosure, that discouraging classrooms warp students’ motivations and active engagement, and that beliefs about whether someone like you belongs in a place like this can dramatically impact whether you participate in the learning around you. And research into the ways that Black, Latinx, LGBTQ+ and other underserved students are punished for being vocal or asking for help is so vast it’s difficult to even know where to start. Here’s one place. Here’s another.

Maybe this is a big beginning for a small post, 30,000 feet up when the only answers I have are small conversations, quiet, iterative. But I am a social scientist at heart and I see learning less as individuated actors making independent motions and more as an ecosystem, all of us coming together and impacting each other in many directions. I think about environments before I think about anything.

And I thought it was useful to start with how I have come to see the action of a student asking for help: courageous, difficult, and precious.

---

Last spring my youngest brother’s college sent them home; in a globally non-unique fashion, his learning experience became disjointed and chaotic. Students navigate and contrast different performance and achievement expectations, but also differing access to information. First the classes were cancelled, then labs reinstated, then the lectures for the labs were cancelled, then the labs were cancelled. Housing was revoked and reissued, cities and universities tossing liabilities back and forth like student lives were so many volleyballs. But my brother lives in Canada, and so he is lucky: cases were low, rent was lowered, spirits were high.

But of course it was scary, and the cracks in the experience emerged swiftly and mercilessly, the domino effects of change that would roll out for any learner undergoing it. Switching from the classroom to video calls in his stuffy small bedroom was depressing, and noisy environments make it hard for him to focus. Absent a unified learning plan, individual faculty made their own and communicated them via haphazard emails and unclear wording. He got instructions to download surveillance software before instructions about whether grading would change to account for the projects they could no longer do. Teachers and people care, but School as an institution seems not to care where students live, or whether they have a room to do their work in — until it occurs to someone in some office of their own that students could use their cramped bedrooms to cheat. Then School demands access, and information, and control. All the things that students are losing, the world’s worst game of no-return volleyball.

I think about that a lot, this year. That as learners, sometimes it feels that we only have bodies that exist when institutions want to control them. That the responsibility to build an environment only goes one way.

---

Because I didn’t go to school for years and years and then clawed my way back into it, I feel like a bit of an expert in being a ghost. I exist only from a certain point of view, in certain systems. When I took a prestigious internship, my first job in tech, I waited patiently in a lobby full of free snacks while HR sought a solution to a combinatorial datapoint their software system would not accept: you couldn’t list a PhD institution, if you didn’t list a high school.

Many learners lead such transient digital lives. There might even be more of us than there are of you. I’ve mentored a lot of students who have spent significant amounts of time out of school, and are trying to get back in. As an education researcher, I often hear people talk about drop-out in education like a single line on a chart, that bad cliff that goes down. But drop-out is really fascinating. It splinters, when you talk to people, into a thousand small thresholds, the trap-doors they walked into. People drop out because a teacher yelled at them, or a law changed around them, or they no longer believed they could do it, or they hated math, or, sometimes, because all of the schools closed. I mentor a lot during normal years — but of course for many of us normal was always a finite quantity, unevenly distributed. It is hard in 2020 to remember being a grad student at a small conference arguing that it was important to study the longitudinal engagement of students in online environments and whether they believed they could succeed, a decade ago. It is hard to remember how I didn’t find funding to explore access, back then. It is harder, in 2020, to wonder if I could have helped prepare us better.

My brother is a better texter than talker, which I take a certain delight in because I taught him to read. It’s nice when we get to see teaching pay off. When his school closed, I couldn’t get him on the phone but I got him on WhatsApp. Some people have doctors in the family to check in on their colds. He has a doctor-of-data to ask, have they turned you into a ghost?

Surveillance software for proctoring can be invasive, and threatening. Across countries and universities, such software demanded access to eye movements, room scans, and head tilts. It made punitive demands: don’t move, don’t drink water, don’t have roommates, don’t be not-white. Don’t ask for help, especially.

The proctoring software gave my brother a panic attack. That didn’t matter, he could handle its crushing impact on his own performance, he said (you learn to be a ghost pretty quickly), but it wasn’t just him. It was all of them, students in cramped bedrooms trying to get by, pretending to be learners, hoping for the small kindness of not being humiliated.

The world is so weird right now, my brother wrote. It’s been intense, living in history. There are students in this class who are going through an incredibly difficult time, and I need to stand up for them.

He wanted to ask the automated proctoring to stop. This software was still under the control of an individual instructor who probably didn’t mean all this harm. So I helped him write a letter, an argument, a research-backed thesis about data privacy. As a tech worker, I knew how to push back on biased software. As a learning scientist, I knew how to channel evidence for just what kind of environment this was creating. As a sister, I was proud, and mad, and in much the same place that I’ve been in since I clawed my way back into the education system, fifteen years ago. Convinced that the only way we make a human environment, is to treat those inside of it like humans.

---

There is something particularly fine-grained about the damage you take when someone makes you feel ashamed of yourself after you ask for help. All kinds of threats can hide underneath instruction. In a world of tech-mediated learning, even small cues about who is allowed to ask for help can become weapons. I got an entire PhD in the disclosure of achievement: and it has taken me this long to decide that it was ok to talk more about not being in school — and what was worse than not being in school. Making it back to school, and feeling unwelcome.

It is hard, in 2020, to have been a student out of school and then to watch so many people talk about what it means to have these students out of school. It doesn’t mean one thing, of course — it is always the splintering, the thresholds, the trap doors. Yet the education conversation has centered on performance, because we have developed little else in the way of a vocabulary around what a learning environment is. But what would it mean to ask less about performance, and more about sustainability and care? There is a paradox in education measurement, and it looks like this: sometimes when learners start to believe in themselves more, they start to look worse — because they experiment, and we don’t usually operationalize this as anything but problematic. They ask questions, and behave in unexpected ways, and yet we design systems around finding and crushing the unexpected.

This approach isn’t going to work when education is disrupted. I know it; I’ve lived it. When we drop out of school, it isn’t just about what breaks but what gets repaired. It isn’t just about treating us like ghosts, alien, our experiences invisible because they’re unexpected. Evaluating learning loss is part of how we deal with this, but it’s such a smaller part than bringing learners back in the first place. It’s about preparing our systems to hear and hold and value those differences, and rising to the challenge of making our education ecosystem the welcoming one we’ve always promised.

This ecosystem is more of a dream than a reality for me. Some learners get to have bodies and classrooms and desks, but so many are muted across video screens, alienated from their own reality down to their eye movements, on guard against hostile software and vicious evaluators. There is no learning without safety.

But there is also hope. Because spending most of your life outside of school also gives you this: the knowledge that learning lives everywhere. I have at least an infinite optimism that someone will always raise their hand, and start asking questions, and because of it, things will get better. In this one class, my brother was one student who got one piece of software removed. Today, a lot of things are broken. They were broken before, but the brokenness has caught up with us, scaled with the force of our institutions, scaled across our shared experience of crisis. And yet in the middle of this, learners persist.

we have to stick together through this kind of stuff, my brother texted.

The borders closed between us during a pandemic that canceled his graduation and my wedding, and neither of us can see our family or friends or coworkers. Nevertheless, we are always part of each other’s ecosystem. We are cues, too, reaching out past screens and texts and fear.

I told him, there are lots of people out here on your side. It is ok to fight back when you think things aren’t fair. Tell me how I can help.

Thanks for listening, he said, you helped already.