University of Maryland Students Develop Image Recognition Software to Scan Patient Receipts
Author: Grace Zhang
What can a small team of Computer Science and Computer Engineering students accomplish in a single college semester? Isabella (Isa) Herold, a senior Computer Engineering student who will be joining IBM as technical consultant after graduation, met with us for an interview to answer this question. Isa and her seven teammates played an integral role in the development of AFFORDABLE’s platform during their senior capstone project.
In the Spring of 2019, AFFORDABLE partnered with Dr. Purtilo at the University of Maryland, College Park to introduce a large software project for a team of Dr. Purtilo’s students. After releasing a quick personality survey, Dr. Purtilo matched Isa with her teammates, and they were assigned the AFFORDABLE project. Since then, Isa’s team began meeting weekly with AFFORDABLE’s founders, Alex Campbell and Sean Pannella to work on their new capstone project– a grant management system for AFFORDABLE’s users and administrators. Users can request Health Utilizing Grants (HUGs) through the portal and submit receipts of the purchases they made with the grants. The administrators can then view those grant requests, verify that transactions made with the grants were appropriately used, and customize the AI image recognition criteria.
To build this system, Isa and her teammates worked together in sub-teams focusing on front-end development, back-end development, and image recognition. They also checked in weekly with Alex and Sean to demo their prototypes throughout the semester and find areas for change and improvement. Isa believes the frequent client meetings were extremely beneficial to the development of the project, as working with Alex and Sean allowed her team to understand how the one project they were working on connects to all the other parts of AFFORDABLE.
Isa’s team worked with many new technologies. Some technologies were particularly challenging; the team used Amazon Rekognition, an image recognition API provided by Amazon Web Services, to enable processing of users’ receipts and verification that their purchases fit in with the realm of their grants. However, when the team first incorporated Amazon Rekognition to their project, it took the system about 15 seconds to process two images, which was longer than they had hoped for. The team then searched for more efficient image recognition software, but none were as reliable as Amazon Rekognition. Eventually, the team found that allocating more memory sufficed for their current use case.
The image recognition team included Ben Haley, Leif Hagen, Kevin Postlethwait, and Rohan Dixit. Isa and Grayson Miller worked on the web interfaces, which include submission of new transactions as well as views of past transactions using React and Express JS. Peijo Yang and Harman Anand worked on the infrastructure side such as hooking up an SQL database and leveraging AWS. They spun up an EC2 to host the webpage, stored scanned receipts on S3, and configured API Gateway to interface Lambda as well as the image recognition software.
All the challenges the team faced and the numerous hours they spent working together accumulated into an extremely rewarding experience. In the final weeks of the semester, the team presented two large demos of their project to their professor. For Isa, this was her favorite part, as “We’ve really seen our project come together […] It’s been very rewarding to see our work pay off.” Now, as the semester ends, Isa and many of her teammates look forward to graduating and beginning their careers, but Isa believes that she and everyone else on her team will continue to help AFFORDABLE in the future.
To Isa personally, AFFORDABLE is especially meaningful, as it was the “first project [she] worked on within her major, school, and internships that will be used to help connect many people, especially those who may need a little bit more with healthcare.” Isa believes that AFFORDABLE is something that will impact the lives of many individuals, so knowing that her team helped to build the platform and being able to say that something they created is helping people will motivate them to remain involved with AFFORDABLE as it continues to grow.
