Kira Systems is a Toronto-based machine learning startup building software that allows users to train models to extract information out of contracts. Primarily designed for lawyers to assist with legal review, Kira has been growing steadily over the past three years and now has 165 employees including an 11 person design and research team.

We interviewed Mary Mikhail, a User Experience Researcher at Kira Systems, to learn more about how they approach user research in a growing startup.

Mary: During mergers and acquisitions, lawyers work on diligence that involves heavy contract review. Kira was initially developed for that use case by Noah Waisberg, who thought there ought to be a better way of doing that.

Behind our technology is the co-founder and CTO, Dr. Alexander Hudek. Although initially developed for a specific use case (M&A due diligence), Kira is now proving to be of value in a multitude of other use cases such as lease abstraction, deal points studies, finance, compliance, contract metadata projects, and more.

Mary: Kira has grown exponentially from the handful of employees working here in 2014, to about 165 people now. As with any quickly growing organization, process is constantly a work in progress. Luckily, design and research have been a core focus of our organization from the get-go. Since I joined just under two years ago, we’ve evolved from a few initial research sessions to hundreds of sessions focused on multiple different features. We also run regular co-creation sessions and workshops with our users.

There have been difficult moments of transition. We went from sitting with developers and directly building things to a more formal product design framework with product managers on the team. One thing we did in the research and design team was to set aside a window of 2 - 4 hours on Mondays—which we called “Meta-Monday”—to work on core design things like our design system and other process stuff.

Mary: We worked on a new feature that enabled users to group documents together based on structure and content. This was challenging because it started with the Machine Learning team that had developed the technology first and we had to evaluate how it could be used. It ended up teaching us a lot about users’ mental models for how they expect to understand an unsupervised learning algorithm.

We heard a lot of feedback from users saying they wanted more information about the algorithm. The reality we uncovered was that it wasn’t about the amount of information shown, but rather the need to feel like the users’ understood how the algorithm grouped things. It was an exercise in absorbing the complexity and presenting the clarity. This work required depth in synthesis and solidified our friendship with ambiguity, and was a great collaboration with our data scientists.

Mary: As an organization, we use Zoom, Slack, Jira, and Notion. In terms of our design research team, we’ve been early adopters of Realtimeboard (now Miro) and Dovetail, and we use Figma, Notion, and InVision. Our frameworks and tool selection have been very fluid as our teams and needs grow.

Mary: Pleasantly, a lot of our research constraints are not due to a lack of design research interest within the organization, but are rather circumstantial.

Security is central to Kira due to the confidentiality that lawyers maintain with their clients, so we can’t actually implement any third-party testing tools anywhere in our app. We aren’t able to ever see our users working in their own instance, and everything that is user-inputted in our system is anonymized to us. This has forced our research approach to generally be creative; from using extrapolations and abstractions, to creating our own testing instance to test features in a live version of the app.

Another constraint is with recruiting. Our users are lawyers so we can’t actually compensate them for research as it’s considered bribing! To overcome that, I’ve crafted personalized gifts (coasters, notebooks, etc…) with the help of a laser cutter at a makerspace. These have shaped the research experience with our respondents.

A more significant byproduct of this constraint was related to documentation. Since we can’t always have too much exposure with lawyers for research, we’ve had to think about how much of the research we can reuse when we do it. The return on investment of the research has helped us become strategic with our documentation. Tagging the data for a specific study versus for future research has played a more prominent role in this space, especially in the early phases of an upcoming feature. There is a trade-off between moving fast but starting from scratch, or moving a little slower but building on the previous research and that’s where we’ve used Dovetail to code the data in a way that allows us to shape our assumptions before launching studies.

Mary: Our research team was previously centralized. As the team expands, researchers are now embedded in our pods. Syncing across these pods is becoming a challenge. We try to focus on our process and work collaboratively across pods to validate and challenge each others’ methods.

Mary: A few years ago, I sketched an inverted pyramid divided into 3 horizontal segments to represent what I look for in a job.

The first and most significant part of it is the implication on my humanity. Does the work that I do expose me to healthy relationships that will challenge me to grow in my interpersonal relationships, emotional intelligence, patience, resilience, conflict resolution, and so on. I do believe that the most important part of what I take away from work ought to be the most transferable. It also feels like there is growth, even in moments when things are challenging.

The second layer has to do with the technical skills used and acquired. Skill stagnation scares me. Thankfully growing and learning in this category comes effortlessly when my mentors at Kira are people that care about the first part of the pyramid. The solidity of the relationships built also inspires me to push myself, and the team, in the second layer.

Lastly and least significant to me is the industry that I work in. My curiosity and desire to learn and grow isn’t confined to the specific industry that I work in, although I tend to favor complex problems to solve. Working on an enterprise solution isn’t necessarily what designers or researchers would think of when they think of ‘sexy’ design challenges, but the ability to see features through from conceptualization to shipping is always exciting. These ideas have legs and giving them the room to grow in stages of ambiguity is hard in the moment, but always rewarding retrospectively.

Mary: Curiosity led me here. My background is in documentary film but the frustration and limitation of the camera challenged me to seek solutions beyond the lens. I ended up doing a very practical program at the Copenhagen Institute of Interaction Design in Denmark, which was a great primer and anchored my desire to pursue research.

My advice for new user researchers is to pursue the right kind of mentor that you can trust and is excited to ‘bet’ on you. Through that close working relationship and oversight of your work, they’ll be able to provide honest and applicable feedback that isn’t always necessarily what you want to hear.

The things that were hardest to hear were always the best pieces of advice that helped me become a sharper researcher, helping me become more critical to uncover all the ways I could be wrong.

Lastly, another piece of advice is to try and be collaborative with other types of researchers. Working with our data scientists has been incredibly enlightening, and ended up leading me to CHIIR this year, which was the best conference I had ever been to. There is a lot to learn from academic research, especially when you work in a space that doesn’t have best practices developed such as the Machine Learning space.