NVIDIA

As part of CS 210, my team and I worked with NVIDIA to build Jet, a multimodal Edge AI classroom assistant that uses voice, computer vision, and real-time student feedback to help instructors adapt their teaching as a lecture unfolds.

A lecturer speaks, and students struggle to simultaneously listen, process, and take notes. While there are many tools that support remote learning, in-person learning still offers limited real-time support for helping students engage with and understand material as it is being taught.

By placing powerful AI compute like the NVIDIA Jetson Orin Nano directly in the classroom, we can create a “supercharged” learning space that listens, sees, and assists in real-time.

Jet supports four core flows across the teacher and student views.

Starting a teacher session

An instructor creates a session that students can join with a code, then uploads their slide deck. Jet reads the slide content and generates comprehension questions from it, which the instructor can review and save to their question bank for use later in the lecture.

Live transcription and generation

Jet transcribes the lecture as it happens, turning speech into a running record of what is being taught. Saying "Hey Jet" lets the instructor generate content into the slides mid-sentence — live code, diagrams, and images that appear on the deck without breaking the flow of the lecture.

Student confusion button

In the student view, anyone can tap a button to flag that they are lost, without raising a hand or interrupting. Those signals aggregate into a confusion meter on the instructor's dashboard, so the room's understanding is visible as a number rather than guessed at from faces.

Hand gestures

The Jetson's camera gives Jet computer vision on the teacher's presentation. It locates the instructor's board and reads their hand gestures. This lets them navigate the deck and move generated content from anywhere in the classroom, so they don't have to walk back and forth to their laptop.

We spoke with three teachers across high school and college settings to understand how they prepare for and run classes, as well as how they perceive AI and its use cases.

Innovating for the future of education

Teachers saw AI as an opportunity to rethink how lessons are created and delivered.

“I think education will be disrupted in an incredibly positive way... you’re kind of exploring the early days of what I think is going to be really meaningful.”

— Mr. B, university lecturer and professor in computer science

Course correcting and generation in real time

Teachers wanted AI to reduce prep work by generating relevant support in the moment.

“Before I introduced anything, going back to the Great Gatsby, I always showed the class pictures. But I’ll spend a lot of time the night before looking for Google images for those pictures. To have that come up automatically, oh my gosh, that’d be incredible.”

— Mr. H, high school English teacher

After speaking with teachers, we brainstormed where we could add value (without reinventing the wheel).

This led us to explore three real-time opportunities:

From there, we scoped the work into sprints on our GitHub board, so each feature had an owner and a deadline before our scheduled demos.

We demoed Jet twice in a Data Science section, where the lead TA used Jet to present his slides to a group of students, who were also onboarded to our webapp.

Our first demo identified key bugs, insights, and opportunities to improve the experience before returning for a second round of testing.

Data Science TA using our Slides Viewer

Students joined the session through a unique code, allowing them to interact with Jet in real time.

Testing Slide Control Gestures

We simplified the gesture system, moving from flat hand gestures to two-finger swiping.

Teacher Session Summary

A confusion timeline, synced with the lecture transcript, shows which parts of the lecture students were most stuck on.