Interview with Dr. Charlene Wong

Researchers (from left) Mike Kaiser, Charlene Wong and Cjloe Vinoya
Researchers (from left) Mike Kaiser, Charlene Wong and Cjloe Vinoya (Photo by Emma Lee/WHYY) Link to WHYY NewsWorks story below.

Charlene Wong is a pediatrician in the Robert Wood Johnson Clinical Scholars program here at Penn.  I spoke with her about her recent study on how young adults use Healthcare.Gov to choose health insurance, and also about the ways in which the WIC was able to support her team in that process.

Q: Hi Charlene.  Tell me about the study you did.

CW: We did a study looking at how young adults are able to navigate and make decisions about the health insurance plans on We were really excited because from the study we were able to make 6 recommendations about how to change and improve to better support people in making informed, smart decisions about health insurance.

Q: So you weren’t looking at what plans were offered—you were really looking at the interface that users were interacting with?

CW: Exactly. There were 27 health insurance plan options that young adults had to choose between. These were all young adults in Philadelphia age 19 to 30, and we were looking to see how is it that they were choosing between those 27 plans: What did they like about the website? What did they not like about the website? How did they think that this website could be improved?

One of the reasons why we were considering these questions in the millennial population is that young adults are uniquely positioned to make thoughtful recommendations on how to change the web-based health insurance marketplace platform, because they’re a generation that’s grown up online – shopping online, rating things online. So, they’re able to give us some really creative suggestions about how to improve this online insurance shopping system.

Q: What was the process you used to do this?

CW: There were two parts to this study. In the first part we brought them in and put them on a laptop computer here at the Vitale Digital Media Lab.  We watched them for 30 minutes as they went through the process of trying to choose a health insurance plan for themselves, and we asked them to go through the process as if they were sitting at home.. These were young adults who were actually looking for health insurance for themselves during the first open enrollment period, so it was not a hypothetical study setting.  While they were navigating online, we asked them to think aloud about what it was they were doing on the screen and what they were thinking about as they saw new plans come up, because we wanted to capture their reactions and thoughts in real time. The great part about the observation period is that we used ScreenFlow software here in Vitale, which enabled simultaneous screen and voice recording and helped us capture some really rich data from this part of the study. Right after we finished the period of observing them on, we interviewed them using a structured interview guide.

Q: And, what did you discover?

CW: One of the fundamental challenges of this process for anyone is that health insurance in the US is complicated. These plans vary across so many different dimensions. But despite these inherent challenges, these young adults identified 6 recommendations that are relatively simple changes to be made to the web based platform that we think will help better support people in making insurance choices. Not necessarily just for young adults in Philadelphia, which are the group that we were looking at, but for people of all ages and all over the country. The recommendations included things that are as simple as giving better explanations for health insurance terms, like deductible and co-insurance, that a lot of the young adults weren’t familiar with. Think about it – anyone would have a hard time choosing a health insurance plan if you didn’t understand those terms because they’re fundamental to processing the different prices displayed when shopping for health insurance.

Other recommendations were making it clearer what health insurance benefits are included in different plans. For example, a lot of the young adults wanted to make sure that preventive care would be affordable in their health insurance plan. Well, as it turns out, preventive care is included in all of the plans for no additional cost because it’s an essential health benefit under the Affordable Care Act.  But that wasn’t obvious when looking at the details about each plan, so their recommendation was to more clearly state these benefits up front when looking through plans. They also again drew on their experience of being very internet savvy, saying things like, we wish there was a better way that we could narrow down our options using checkboxes or sliding bars where you could indicate price ranges that you would be willing to pay for the premium or deductible. You’d be able to check off what important insurance benefits you’re looking for: do you want dental coverage?  How about mental health coverage? They pointed to other websites like Amazon or where these sorts of tools are readily available.

Q: Have you been in touch with the folks running the website to give them your recommendations? If so, what have they said?

CW: We have sent our results to contacts in the Office of Health Reform because we know they are redesigning and trying to improve the website before the next open enrollment period in November of this year.  We hope to have some in-person, or at least phone briefings with them to give them even more information about our recommendations. But certainly, we’re also very thrilled that these recommendations are now publicly available in the article and also with the press coverage that’s come from it.

Q: Where was it published, and what other press have you gotten about the study?

CW: The study article was published in the journal Annals of Internal Medicine. I  also published an Op-Ed in the Philadelphia Inquirer with David Asch and Raina Merchant that summarized the results of our study. The Inquirer also wrote a separate article on the study because we were highlighting how local young Philadelphians who participated in this study made recommendations they may change the way people all over the country are going to experience selecting health insurance. The local NPR affiliate WHYY and Knowledge@Wharton Sirius XM radio ran segments on our study. The Leonard Davis Institute has also covered and helped disseminate our findings.

Q: Why did you choose this topic?

CW: At the beginning, when the health insurance market place was opening, a lot of the focus was on’s technical issues and how the website was failing. But I started thinking about how difficult it’s been for me in the past as a doctor–someone who has a lot of experience with health insurance –to choose insurance for myself or my family. And then, as a pediatrician specializing in adolescent and young adult medicine, I was thinking about my young adult patients and how they are often purchasing health insurance for the first time.  And I wondered how is they were going to manage the process of navigating selecting a health insurance plan, and in particular on this brand new platform that had previously never been tested in this sort of broad sense. So that’s really where the study idea came from. It became clear early on that one of the biggest impacts of the study was going to be that these young adults had great ideas about how to improve the website.  That’s how we targeted getting these study recommendations out first—and quickly, because we wanted to make sure there’d be time for these recommendations to reach the people in Washington who are redesigning the website.

Q: Tell me what resources you used in the Information Commons and the Lab and the Library, and how we were able to help you through this process?

CW: There was no one on our research team–and we have a pretty large, multi-disciplinary senior research team–that had the technological expertise to recommend how to do screen recording on that we wanted. So meeting with you and Anu from the Weigle Information Commons and the Vitale Digital Media Lab was instrumental in making the study feasible for us.  Otherwise, we weren’t sure how we were going to capture the data that we needed. WIC also provided us with essential equipment—a laptop, an audio recorder, and a microphone in addition to the screen capture software we used.  You all also  taught us how to use it all, because no one on our team had experience with these tools. On top of that, we were also able to do the actual study, here, in the WIC.  The location was accessible and easy to find for the local young Philadelphians who weren’t affiliated with the university. You all also helped use arrange for weekend and evening access for study participants, which was really helpful since many of them work during the day.  This flexibility was important for reaching our study recruitment goals.

Q: And you used study rooms at WIC?

CW: We used the study rooms here at WIC, which again was really fantastic because we wanted a quiet, private space for the participants to be talking to us about what they thought about the website.

Q: I understand you used a program called NVivo in this study.  Can you tell me about that?

CW: NVivo is a software program used to analyze qualitative research. We’re using it for the study transcripts, which we read through and then develop what are called nodes or themes. The software allows you to rearrange qualitative data transcripts into these themes. That way, when you’re analyzing your data, you can say, “62% of participants discussed” this particular theme.  We’re also using NVivo for our screen recordings because it can store video and time-stamped video transcription. That way we’re analyzing all of our data, videos, audio transcripts, all within this one program. It’s been great.

Q: Where do you use it on campus?

CW: We use NVivo in several places.  It’s available here in the WIC. We also have access to it in the Robert Wood Johnson Foundation Clinical Scholars Program office and the Penn Medicine Mixed Methods Research Lab.  But WIC has the most available computers with this software, so we have done much of our analysis here.

Q: How did you find Cjloe Vinoya, who worked with you on this study?

CW: Cjloe was our research coordinator, and the two of us did all of the study sessions.  She had worked as a SUMR Scholar, who are undergraduates who get paired with health researchers at Penn. She did such a phenomenal job in that program that she was asked to stay on with her principal investigator at the time, Karin Rhodes, and at the Leonard Davis Institute (LDI).  When I was looking for a research coordinator, she was very highly recommended to me by LDI staff.  And it worked out so well. She has such great rapport with participants, which was important for this qualitative study. Everyone really enjoyed working with her, and she’s incredibly organized.

Q: So what’s next?

CW: We are still in the process of analyzing more of the study data. This article that came out was just a brief article with the 6 recommendations. We have a lot more information to share about young adult’s perspectives on health insurance. We also have data on which plans they actually selected and how satisfied they were with their decisions, so those analyses are still ongoing. Then, as a physician, what I’m also very interested in is looking at how young adults are planning to use their health insurance–which we explored a little bit in this study.  And then trying to identify how to better help them use their health insurance to be healthy.

On top of that, we partnered with a national organization called Young Invincibles, which works on issues affecting young adults including healthcare.  In the study, we identified some knowledge gaps around health insurance, for example health insurance terminology literacy . So they’re considering developing some web based, more accessible explanations for young adults or any adults who aren’t as familiar with concepts like ‘deductible’. If you haven’t ever had to pay the deductible before, it’s a little bit complicated to explain just using a glossary definition. In addition, the young adults wanted more examples  about how to choose between all these different health insurance plans or tools that help them make a decision because there was so much information to process. These are issues that may be the focus of future projects coming out of this work.

Q: Thanks so much for talking with me.

CW: You’re welcome!


Annals of Internal Medicine (via Penn Libraries. The Experience of Young Adults on Suggestions for Improvement. August 5, 2014)

Tech-savvy subjects test website, advise changes (Philadelphia Inquirer)

Op-Ed about the study (Philadelphia Inquirer)

Study of young Philadelphians’ experience on health insurance site may guide improvements (WHYY NewsWorks)


1 thought on “Interview with Dr. Charlene Wong

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s