Kate Krontiris
‘Wait, How Does This Work in Government?’: Kate Krontiris on Piloting User Research within the Executive Branch
| Kate Krontiris was a user researcher during the early days of the U.S. Digital Service, focused on the immigrant visa process. Previously, she held roles at Google and the U.S. Department of State. |
Kate Krontiris contributed user research to the early U.S. Digital Service, helping adapt user research to the executive branch. Her user journey interviews with visa applicants in Montreal and Santo Domingo motivated engineers within the government to place an added value on design and research.
Below, Kate speaks with USDS founding members Kathy Pham and Emily Tavoulareas about the unsung heroes of USDS, the importance of diverse skill sets within a team, the complications of the Paperwork Reduction Act, and more.
June 6, 2019
Emily Tavoulareas:
Kate, tell us how you ended up at USDS.
Kate Krontiris:
Mollie Ruskin. Mollie and I worked together immediately before she left to become a Presidential Innovation Fellow. And as the office was shaping itself, I knew you all were putting together ways of working. I had left the organization that we were both working at and started my own portfolio, and so was very flexible in terms of my ability to take projects on. Mollie was the connection.
There was paperwork that I had to fill out, and background checks, and I came to DC to meet people before it was all done. But it was basically like, “Just come help us.”
Emily:
When did you start?
Kate:
I probably started in the fall of 2014.
Emily:
Tell me about your first day.
Kate:
My role was very targeted — coming into DC for specific meetings and then getting out. I remember coming into Jackson Place and being like, “What the heck is this place? It’s awesome.” There were all these cool people. I remember having a very early meeting with Viv Graubard at the State Department about the project, but I always felt more like a consultant than a team member. So my memories are mostly about just coming in, having meetings, and then heading back.
Emily:
Do you remember how the team was set up at that time?
Kate:
I vaguely knew job descriptions and that there was a lot of work people were doing. I also have this memory that it was difficult to work together as a team. I remember a lot of process changes, but I wasn’t too connected to the guts of how it was working: the interpersonal relationships, what positions people were taking, what demands were being asked of them.
Emily:
Tell me your memory of USDS’s founding. If someone asks, “How did USDS happen?” how would you answer?
Kate:
Basically there was this whole initiative, the Affordable Care Act, to get Americans health insurance. It was dependent on healthcare.gov, a technical platform that was the implementation of the policy. As we all remember, that was a huge failure, and so the Obama administration brought in a whole bunch of people to help fix it, historically drawing from tech sectors in a way that really hadn’t happened before.
Once they saw what was possible from that collaboration between government and private sector, they realized there are tons of related policy challenges across government that could use the same expertise. So they started to build out the United States Digital Service, an effort out of the White House, that could work on similar policy challenges where there’s a real nugget of technology that affects the implementation of the policy or the program. That’s the founding story that I learned.
Emily:
It’s interesting. Part of what we’re doing here is collecting different perspectives, and we’ve learned that there were a lot of very different vantage points on how this all happened. You being remote gives you a different vantage point on the founding story. Do you remember when you first came to DC and interacted with the team in person?
Kate:
The first time was administrative, so it was just meeting other members of the team, working out logistical stuff. Then I was pretty much right away put on this project that Viv was leading with others, around modernizing the immigrant visa.
Emily:
What was your impression of the way the team was structured at that point? The team was in startup mode at the time you joined, and I’m curious to understand your perspective of the organization. How do you remember it?
Kate:
That it was very startup-y, that they were trying to figure out how to work within the rules of the bureaucracy around hiring. There were constant questions of, “Wait, how does this work in government?” At some point, being able to do user research comes against the Paperwork Reduction Act. And it just seemed, because of the fast-paced and high-profile nature of the work, that people had a lot of emotions about things, and there were a lot of disagreements about how to do things and people’s working styles. Like any high pressure working environment, it was fractious. It was difficult. The founding team was just working out how to do this thing and how to work together as a lot of Type A people. That was my impression.
Emily:
Do you have a recollection of any of the issues that created the most friction within the team? Or were really big challenges that the team had to solve when facing the bureaucracy, in order to work the way it wanted to?
Kate:
It seems like diversity in recruiting was a big issue. And different leadership was an issue. When engaging with the bureaucracy, there was also potentially this hierarchy of skills. The skills that are about action being strategic and getting the bureaucracy to move were undervalued a little bit initially, compared to “go fix a website” skills.
But also everybody’s doing everything. It felt internally like, “This is new. People haven’t been doing this.”
Kathy:
It was so interesting to see the types of things people put in the intake form for this oral history project… what everyone wanted to talk about in these interviews. For you, it was your work shadowing people who were applying for immigrant visas to the U.S., to learn how to make the process smoother, including for the Dominican Republic and Canada. It was the first time the State Department had done this kind of user journey shadowing. Can you talk more about the decision to get onto that project? And also what that was like — your day-to-day, how long that took, what time period, which countries you went to?
Kate:
That project was fascinating on so many levels. In graduate school I actually worked at the State Department for a summer, a couple years prior. So I was sort of familiar with how the State Department worked. I’d worked for Alec Ross and Ben Scott. I really admire the way that Ben Scott did his best to work the State Department bureaucracy when he was there. And Alec had a different kind of public figure-y approach to things. I learned a lot about how the department works. And it just happened that was the project I got tasked to when I joined USDS.
I remember one of my early meetings with Viv on this issue. And the issue, just to remind you, was that the process of getting an immigrant visa to the United States is extremely challenging. We want to modernize it, and the State Department had been in the process of modernization for quite a long time.
I remember a meeting we had with the State Department point people, and Viv being super good at keeping the meeting going, and her being this young woman in a meeting with a much older, more senior State Department woman, being like, “What are the things that are problems right now? Let me figure out how to work with that, and what else can we unblock.” I’m sitting there listening and learning about the process. That was an early meeting that was very revealing, just in terms of the level of access that this team from the White House was getting.
We had to go to Maryland to meet them. They were having this meeting to walk us through the technical systems and the work that they’d done previously. I remember being viewed among this group as the person from the White House, and not having realized I was going to be viewed that way. I was thinking, “I’m a consulting user researcher, I’m going out to get information. I’m not the boss coming in.” But it was a room packed full of people in suits anticipating a visit from the White House. I don’t know what had been communicated to them.
There were more planning and preparation meetings, and then the State Department paid for me to go to Santo Domingo and Montreal. The first trip was to Montreal, and it was very cold. The consular people in DC were having lots of pre-planning conversations with their counterparts in the consulate or the embassy to plan this trip. Once I got there, I did a whole bunch of meetings with embassy staff internally.
In Montreal, we talked about following a couple of people who’d come for their immigrant visa appointments. Then basically the next day I got in line with people in the freezing Montreal cold to go have their visa interview and started talking to some of them.
This one young woman was pursuing a K-1 visa, for marriage, and that’s not technically an immigrant visa. But it was a starting point to understand her experience so far. I sat with her through the course of coming into the building, getting screened, sitting in the waiting room. I went up to the window with her as she had her interview and the decision was made, and then we debriefed. Then I’d go back in the waiting room and try to find somebody else who was interested in letting me talk to them. I did this a few times and got a couple of different perspectives from different people.
Kathy:
Before you got there, was there any kind of a process or approval? Or was it like, “Okay, we’re just going to do this?”
Kate:
There was a lot. And Viv, as far as I can tell, managed all of those discussions. But that doesn’t mean they had an idea of what I was actually going to be doing.
One of the reasons they had me go to Montreal was because it’s a slightly more privileged population of people seeking a visa; snowbirds. But in Santo Domingo, it’s different. Many of them have family in the United States, and their options are just not great in the Dominican Republic. There’s more of what the State Department calls “fraud.” In some cases maybe it’s actual fraud, and in other cases it may just be due to illiteracy, people fundamentally don’t get the processes. They’re confusing, and they’re reliant on explainers.
There’s a whole ecosystem of third-party actors who facilitate people getting these visas that the State Department both has to contend with and also hates. Those third-party actors often have either false information or just bad information about the process. And yet people are absolutely reliant on them to understand because again, literacy levels are low.
We had to think carefully about how we were recruiting people to participate in the user experience interviews. In the case of Montreal, we didn’t try to reach out to people ahead of time who had made immigrant visa applications to the U.S. I just met people who were in line for the visa interview. But for Santo Domingo, we wanted to be able to go a little bit earlier in the journey — not just when they arrive for the visa interview — to try and get a sense of what their experiences have been. In some cases, five or seven years waiting.
That was fascinating. They were incredible stories, and it was thanks to Viv, because I don’t speak good enough Spanish. And this woman who was the foreign service officer, who’s a fluent Spanish speaker, was my point of contact for calling people. We pulled a list of people who’d made immigrant visa applications. I also went to see the actual physical files of immigrant visas at this warehouse in New Hampshire.
We got a list of people that we could call, and Viv called 45 of them one weekend, and this other woman called another 45 of them: “I’m actually calling from the embassy in Santo Domingo. We’re doing this project to try and understand customer service. Would you be interested in talking with a researcher? It’s not going to affect your case positively or negatively.” That’s a standard user research recruiting approach — but for years, the embassy in Santo Domingo had been sending out radio and other kinds of messaging that said, “This embassy will never contact you on the phone about your Visa application. This is what scammers do. There are scammers out there that are trying to get money from you to apply for visas and then never do anything for you. Don’t listen to anybody that calls.” And yet here we were making these phone calls: “Actually we’re from the embassy and we want to talk to you about your visa application. We’re trying to see if the process could be better.”
Kathy:
God. Compare that to the Montreal snowbird story.
Kate:
It was a totally, totally different thing. I would repeat again and again, “This is not going to have any bearing on your application, either positive or negative. This isn’t a sign that you’re going to get your visa.” It’s really hard to tell somebody that. But fortunately, the foreign service officer who I was with, who was doing a lot of the translating, was very clear about that. And she put herself in a position, because people recognize their foreign service officers. But it was a thing that she did in service of trying to improve the process.
Some people tried to save money by sleeping on the bus overnight. When they arrive, they’re really tired. Other people are petrified that they’re going to somehow miss the interview, even though they’re in the room. I remember hearing a story of this mom who had two kids, and the kids basically wet their pants because she refused to bring them to the bathroom so that she wouldn’t miss being called for her interview. You could only feel that level of tension by physically being there.
I also got a chance to workshop in Santo Domingo. After we’d finished some of our user research, I shared what I learned with a combination of local and locally-employed people and foreign service officers, including some of the higher level counselor staff. I wanted to focus group what I learned, to get the perspective of the people who process all of this on the other side of the glass. That was a fascinating opportunity. The people in that meeting were like, “What the fuck is this?” They were really surprised that their opinions were being sought.
Kathy:
This gets to the heart of why we’re doing this, and I love it. Let’s wrap up this story, and then I have a follow up question.
Kate:
Coming back, the whole purpose was to be able to share that user experience with decision makers in DC, such that we could put out this memorandum that the President had requested on how to improve the immigrant visa experience. The user research helped give us some fundamental, guiding principles for the overall memorandum, although Viv and others were then responsible for combining that with all of the work that our colleagues had done on the tech side — to understand this horribly arcane technical system that the State Department uses to manage visa information.
There was very much an awareness that this was business improvement. Customer experience improvement efforts had been undertaken in the State Department, but this way of shadowing the actual people whose lives are in question in this visa process was something that hadn’t really happened before. And to their credit, the State Department people were like, “We’re not experts in this. We don’t know how to do this. And we’re going to put enough trust in you to bring somebody who’s not one of our own in to do this with us, even though there are risks associated with it.” So it was a managed process. There was a lot of consultation. But they deserve credit for being open to a process that is not baked into the way the State Department usually does things.
Emily:
How do you think the State Department got there? Had they been feeling that way from the beginning? Or is that something that changed along the way? That willingness and openness to do things differently is a really big change to operations and their perspective. And I’m curious: Is it something they intuitively understood the value of, or did it take something to get there?
Kate:
They were aware that immigration was an important priority for the administration, and the State Department is uniquely responsible for representing the country abroad. So maybe they are uniquely attuned to what the President wants. Number one, it was clear that modernizing the immigrant visa process — although it had been an ongoing thing for a while — was important to this administration, and thus they needed to prioritize it.
I also think Viv deserves a lot of credit. Again, there’s this undiscussed labor that happened to get people to trust each other and work together. She was able to facilitate a series of conversations, often among people within the State Department who didn’t necessarily agree with each other. I don’t know that she got them to see eye-to-eye about everything, but she managed to make this project work and get their involvement and buy in. The fact that the State Department paid for me to go do this research is an indication of the power of the moment and those skills.
So I don’t know exactly what that evolution was like for them. But I felt like we were in the midst of helping bring them along to a different new way of thinking.
Emily:
Do you think that was a temporary, momentary change? Or do you feel like they really came away from that experience viewing the work and their jobs differently? For the medium-haul, at least?
Kate:
I don’t know that they viewed their jobs differently, because many of them had been working on this as a major part of their jobs before we ever showed up. But I do think that they had an appreciation for the methodology and why it’s different from what their working groups yielded in the past.
If anything, it exposed a lot of problems internally, which is another reason to give them credit — because they probably knew that that would come up. But then they have material and information to act on, to change things. My hope is that it was beneficial for them in some different ways.
It’s so dependent on the personalities of the people involved. I have a highly refined sense of emotional intelligence — that’s just a skill I bring into the work I do. I can read a room and know how to communicate information in a way that feels productive and not accusatory. But not everybody has that skill; people have very different kinds of skills. Viv is also really good at that. Maybe women are just really good at that. There’s a way of managing the process where it could have really gone off the rails if you had people who just didn’t have that kind of intelligence.
It also mattered that the people we were working with internally were senior enough to recognize that even if they weren’t personally interested or invested in change, they needed to get on board because this was a presidential priority. So we were taken seriously.
Emily:
You’ve mentioned the presidency and the weight of the White House in doing the work, and also in the way people perceive and prioritize the work and you as individuals, and allow you into the room in a different way. And I’m curious: What role do you think that played? More specifically, do you think the work would’ve been possible without that?
Kate:
I can’t answer the counterfactual, but I did recognize it as something that’s very different from my usual working environments. I’ve done similar things in a lot of different contexts, and usually the user researcher is not taken that seriously. But in this case, I was always viewed as coming from the White House or in some cases the Office of Management and Budget (OMB), which scares people — because OMB was associated with, “Are you using your money the way you’re supposed to be using your money?”
I think people took us seriously in part because of that positioning. And I’m glad we had that positioning, because things open up a little bit more quickly. In terms of what it would’ve looked like to not have that, I guess we should ask 18F, because they probably had to do similar stuff but didn’t get to say, “The President thinks it’s important.”
Kathy:
What are the things you left behind that really set the foundation for how USDS operates? Was there a conscious effort to do that? Did you see the work laying a foundation for some of the ways USDS continues to operate?
Kate:
I never saw any of that, and that’s probably a useful thing in and of itself, and a function of having this kind of consultant relationship to the team. I don’t even really know what happened as a result of those recommendations we made. I did present what we did back internally to the rest of the team, and then we had an opportunity to present to the Domestic Policy Council — again, through Viv and Cecilia and everybody. There were some immigration advocates in the room for that presentation. That felt really important in terms of sharing what we had learned, and demonstrated how the U.S. government is listening to people in a different kind of way.
I then contributed to a couple of other projects, none of which turned into a full-fledged thing. But there was some work that Ginny Hunt was leading around social service programs. But I don’t know the answer to your question. I’m not sure foundationally what it contributed to.
Kathy:
Coming in, I knew that we sent someone somewhere in the world outside of our comfort bubble to do this work. And then I also knew that the VA team was doing this kind of user research work. It was referenced multiple times with different people around USDS — going where the work is. It made it so that we never thought not to do that. It was just always something that we did. So your work there definitely made an impact.
Kate:
That’s really important. I’m glad to know that. And again, Viv deserves a lot of credit for that because she was the one who hustled that whole thing together. I know other people were doing similar things with different agencies, but I hope there’s an unsung heroes and heroines chapter.
Emily:
In thinking about the legacy of your work at USDS: for what it’s worth, one of my personal takeaways from USDS was seeing programmers change over the years in relation to their prioritization of design and research. It’s like they realized, “Oh wow, this is really, really valuable to what we do. It makes my job easier, better.”
Kate:
One of my strongest memories of that time was the ability to learn from people whose skill sets I didn’t have a ton of exposure to previously, and being really impressed by the combination. In the bigger picture, what is everybody doing with all of that now? What have we gone on to do as a result of having that foundational experience located deeply in the public interest? We’re not just talking about user experience researchers and technologists from IBM or Facebook or whatever, working on commercial products. We’re talking about systemic public problems.
Kathy:
I can’t wait for the end of the movie part, “where these people are now.”
Emily:
It has been incredibly, incredibly powerful to see where people land — where people have gone, what they’ve been doing, and then how their perspectives have changed over the years. We could have endeavored to do this at several points, but it’s easy for me to say now’s the right time. There’s value in having had time pass to this degree — not too long, but long enough to have some distance.
Kathy:
Kate, thank you.
Kate:
You’re welcome. It’s been my pleasure.