It’s a software application, built with openFrameworks, that uses computer vision (OpenCV) and convolutional neural networks (ofxCcv) to analyze a picture of a piece of paper where instruments have been hand drawn, including bass guitars, saxophones, keyboards, and drums.
The classifications are made using a convolutional neural network trained on ImageNet, and sends OSC messages to Ableton Live, launching various clips playing the music for each instrument.
The software will be made available at https://github.com/ml4a/ml4a-ofx
Andreas Refsgaard and Gene Kogan are artists, programmers, and interaction designers, working with machine learning to crx-eate real-time tools for artistic and musical exx-pression.
Seoul National University
AI Scheduler: Learn your life, Build your life
Automatic scheduling / Daily life pattern / Deep learning / Inverse reinforcement learning / Wearable device
컨셉 및 기획의도
In this project, we explore the possibility of an AI assistant that will have a part in your life in the future. Being able to predict your daily life patterns is an obvious necessary function of an AI assistant. In this respect, we focus on the theoretical issue of how to learn the daily life pattern of a person. For this hackathon, we developed a system which can automatically recognize the current activity of a user and learn activity patterns of the user, then predict future activity sequence.
From wearable camera and smart phone data, the system can automatically recognizes user’s current activity, time and location information. For this part, the visual recognition API of Watson and several machine learning algorithms were used. Based on this information, the system learns the patterns of user’s daily life. We devised the learning algorithm based on ‘inverse reinforcement learning’ theory so that the life pattern of the user can be learned properly. Then, the system can generate life pattern of future. Also, as a scheduler, it is desirable for the system to interact with the user and reflect the user’s intention. Therefore, we developed a web-based interface which can interact with the user by natural language. For this, IBM Watson’s conversion service alongside with text-to-speech/speech-to-text was used.
TensorFlow, VGG Net, wearable camera, web-based interface (IBM Bluemix Conversation Service, speech to Text Service, Text to Speech Service, Node-Red).
We are graduate students at the Seoul National University, Biointelligence Laboratory. We are interested in the study of artificial intelligence and machine learning on the basis of biological and bio-inspired information technologies, and its application to real-world problems.
Nabi E.I.Lab (Emotional Intelligence Laboratory)
A.I interactive therapy
Artificial Intelligence, Art color therapy(CRR TEST), New media art, Interaction, IBM Watson
컨셉 및 기획의도
A.I. interactive therapy attempts to analyze human psychology and emotion through artificial intelligence. This artificial intelligence system is an interaction type in which the client conducts psychological counseling through direct physical behavior. This system is based on a creative approach to the imagery of art therapy. It is also designed to analyze the emotional stability and inner aspect of human psychology by making full use of the interactive characteristics and visual effects of new media art.
A.I Interactive therapy는 인공지능을 통해 인간의 심리와 감성에 대한 분석을 시도한다. 이 인공지능 시스템은 내담자가 직접적인 신체적 행위를 통해 심리 상담을 진행하는 인터랙션 타입이다. 이 시스템은 아트 테라피의 창조적 심상에 대한 접근을 바탕으로 한다. 또한 뉴 미디어 아트의 인터랙티브적 특성과 시각적 효과를 최대한 활용하여, 인간의 심리에 정서적 안정과 내면을 분석하기 위해 고안되었다.
This project is crx-eated based on CRR TEST which is a color psychological analysis method. The test analyzes the mental state of the subject by selecting three out of eight plane figures in order. This project has been completed by grafting artificial intelligence onto this method. First, a UI environment has been made by an application for vertical projection on the floor. A kinetic camera has been installed to allow the user to experience the UI environment in real time so that tracking of human movements can be made for choosing movement values and interacting to the values. A voice feedback system has been constructed for processing specific results and overall operation of the system. This proceeds with IBM Watson's conversation API. For this, a sort of chatbot system is employed, and voice feedback technology makes decision on the progress of the process according to IBM Watson's STT / TTS. The calculation of the result value based on the final selection is also made through artificial intelligence. The language for this application is Python.
이 프로젝트는 색채심리분석법인 CRR TEST를 바탕으로 만들어 졌다. 이 테스트는 8개의 도형 중 3가지 도형을 순서대로 선택함에 따라 대상의 심리상태를 분석한다. 이 테스트를 인공지능과 접목하여 프로젝트를 완성하였다. 먼저 수직 프로젝션을 통해 바닥에 투사되어 보여지는 어플리케이션을 UI 환경으로 구성하였다. UI 환경을 실제 내담자 즉 사용자가 체험하도록 키넥트 카메라가 설치되어 사람의 움직임을 트래킹하여 이동값에 따른 선택과 인터랙션이 가능하도록 설계하였다. 특정 결과와 전체적인 시스템의 운영은 음성 피드백 시스템으로 구성된다. 이는 ibm watson의 conversation api를 통해 진행된다. 일종의 챗봇 시스템으로 구성하되 사용자의 음성 피드백 기술은 ibm watson의 STT/TTS 에 따라 과정의 진행여부를 판단한다. 최종적인 선택에 따른 결과값의 산출 또한 인공지능을 통해 발현된다. 이 어플리케이션의 구성 언어는 Python이다.
Hardware : Projector, Kinect v2 camera, Mac pc, Speaker
Software : IBM Watson api, Python, Pykinect2
E.I. Lab at the Art center Nabi is a creative production laboratory that researches and tests the contact points of art and technology. E.I. Lab stands for Emotional Intelligence Laboratory and focuses on creating new contents through emotional approach and technical research. The members are New media artist Youngkak Cho, Software developer Youngtak Cho, Designer Junghwan Kim and Interaction designer Yumi Yu. The lab is currently researching and developing fusion projects based on robotics and artificial intelligence technologies.
Georgia Institute of Technology
Continuous Robotic Finger Control for Piano Performance
로보틱스, 딥러닝, 머신러닝, 컴퓨터비전, 로봇뮤지션
컨셉 및 기획의도
This project uses deep neural networks to predict fine-grained finger movements based on muscle movement data from the forearm. In particular, we look at the fine motor skill of playing the piano, which requires dexterity and continuous, rather than discrete, finger movements. While this project is most directly applicable to giving musicians with amputations their musical ability back through smarter prosthetics, the demonstration of successful fine motor skills from our deep learning technique allows for promising applications of this method and our novel sensor throughout the medical and prosthetic fields.
The final deep learning techniques used to successfully demonstrate the continuous robotic finger control was a four lax-yer fully connected network followed by cosine similarity and a softmax loss. Images were normalized before input and batch normalization was used to smooth out the regression results. In post-processing, noisy regression results were further smoothed with a filtering step. To implement this network and run our prior experiments, we used Tensorflow, Torch, and pre-trained networks from Inception to construct our deep networks.
Deep learning libraries: TensorFlow, Inception, Torch
Data collection: glove bend sensor, MIDI output from keyboard, muscle sensor
Mason Bretan and Si Chen are PhD students working in robotic musicianship and computer vision, respectively. Gil Weinberg is a Professor in the School of Music and the founding director of the Center for Music Technology. They are based at the Georgia Institute of Technology, within the Center for Music Technology and the College of Computing, School of Interactive Computing.