While implementing our apps, we dealt with lag issues caused by large API requests. From a broad perspective, the coordinate information and ML features would be sent to the server via POST request, the server would process and store the necessary gesture and hand position coordinates, and finally the GUI would use a GET request to receive the necessary information. Since UltraLeap required Python 2.7 and GUI required Python 3.7, we used a Flask server to act as an intermediary between the two. Secondly, Python and Leap Motion can be easily integrated into QML. One, QML can run as an app, independent from an internet browser. After a few weeks of research, we decided on QML for two reasons. Our first major challenge was determining which programming language we should use for the front end. Link for NYT COVID-19 datasets Challenges we ran into To optimize our initialization time, we used the Python threading library. We also used Blender to render a 3D video loop of blue electricity to serve as the background.ĭue to the nature of Jarvis, many API requests are needed to initialize Jarvis. We used Figma to design all of Jarvis’s buttons and frames. Jarvis’ GUI was made primarily in QML, due to the language’s compatibility with a Python library, PySide2, a Python wrapper for the C++ Library Qt. In addition, we used various Python libraries such as BeautifulSoup and PySpellChecker to create an auto-corrector for the Corona app. Our time app required the Python time library. csv files provided by Johns Hopkins, New York Times and Data Package Core Datasets on Github*. To create the Corona app, we used Pandas to pull data from. Specifically, the APIs obtained data from Spotify, Gmail, Google Calendar and OpenWeatherMap. Taking these coordinates, we created an LSTM model in PyTorch to act as a classifier for various hand gestures.įor the backend of our apps, we used APIs to retrieve data necessary for Jarvis’ operation. With UltraLeap’s built in library, we were able to track the coordinate information of various joints, fingers, etc. How we built itįor our hand motion detection, we used a Leap Motion Controller. Jarvis allows its users easy access to these necessities through mouseless control. Given the current pandemic, we included a COVID-19 widget to inform users about cases and trends in counties and countries. Jarvis acts primarily as a personal assistant! Jarvis can be used to access the modern day necessities: emails, music, time, and calendar. We hoped Jarvis 2.0 would increase accessibility to technology, as well as change the way we interact with our machines. We wanted to create a way for users to easily access and interact with their favorite applications in one dashboard without having to touch their mouse. Iron Man’s AI assistant, Jarvis, in the Marvel Cinematic Universe (MCU) was the main inspiration for our product.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |