The Research Phase

In November 2018, the user research phase began.

The research phase of the project, conducted in parallel with the scaffolding of the development project, consisted interviews and diary studies run with staff from both WECCC, and staff working at the Population Health Solutions Lab at Mount Sinai Hospital. Research continued to net actionable results until the end of the project via feedback prompts baked into the application.

The interviews

The interviews with WECCC & Mount Sinai staff ranged anywhere from 30-70 minutes, and were based on an in-depth discussion guide (and statement of informed consent) developed in-house.

The interviews were transcribed for documentation and review.

The learnings from these interviews were invaluable, as we gained insights into the survey process, intra- and inter-agency communication, tools, and likes & dislikes from individuals in different organizations in contact with different communities.

Takeaways from the interviews:

• Visual representation of data was in high demand
• Organizational bottlenecks in data access could have web-based solutions
• In-person surveys are considered an important part of the care cycle
• Pen-and-paper input is still alive and well, and not necessarily considered to be a barrier to care
• A pain point was using personal cellular data plans

Takeaways from the interviews (continued):

• Reminders, or 'ticklers' facilitate the check-in process
• A non-linear approach to information gathering is helpful in the survey process
• It is not uncommon to have paper copies of surveys that don't align with digital inputs
• OSCAR is a major pain point
• Comfort levels with technology vary widely, but the willingness to learn was consistent. Good documentation and a non-technical user's manual were in demand

This lead us to several conclusions:

• An accessible API would be valuable
• An application optimized for low data consumption could relieve stress when using personal data, or using public wifi.
• An application should be mobile responsive, and work on-site (should a surveyor use a tablet or phone, but also work as a post hoc data entry tool should surveyors opt for pen-and-paper while conducting a survey

The diary study

A diary study was conducted with WECCC staff over the course of two weeks in February 2019.

In the responses, we saw reinforcement of the learnings from the interviews, particularly with OSCAR. We also saw multiple incidents where the office network had technical issues, and where the existing survey tool caused user errors due to a lack of system feedback.

The biggest takeaway from the diary study was the amount of time consumed by manually entering survey data into an unpredictable system.

This lead us to several conclusions:

• Circumventing OSCAR would be a priority
• Creating shortcuts for "power users" could have big returns in efficiency
• A web-based solution would be resilient when encountering network issues
• A progressive web app, capable of saving data offline and updating when connectivity was resumed, could be a worthwhile solution

Soliciting feedback

Once the application was ready for user testing, we enabled a feedback prompt within the application, giving users a direct line to our development team.

This did not yield any major research findings, but did serve to keep us in touch with staff, and allowed us to patch bugs quickly.