Week 10 – generative music from Melody_RNN

I played with MelodyRNN a little bit

step 1. Convert MIDI files into NoteSequences.

MINE:

convert_dir_to_note_sequences \

  –input_dir=clean_midi/U2 \

  –output_file=notesequences.tfrecord \

  –recursive

2. Convert NoteSequences to SequenceExamples.

melody_rnn_create_dataset \
--config='lookback_rnn' \
--input=notesequences.tfrecord \
--output_dir=./sequence_examples \
--eval_ratio=0.10

3. Train Model.

melody_rnn_train \

–config=lookback_rnn \

–run_dir=/tmp/melody_rnn/logdir/run1 \

–sequence_example_file=./sequence_examples/training_melodies.tfrecord \

–hparams=”batch_size=64,rnn_layer_sizes=[64,64]” \

–num_training_steps=1000

4 open the run1 file

5 Generate some music

melody_rnn_generate \

–config=lookback_rnn \

–run_dir=/tmp/melody_rnn/logdir/run1 \

–output_dir=./generated/melody_rnn/U2/ \

–num_outputs=10 \

–num_steps=128 \

–hparams=”batch_size=64,rnn_layer_sizes=[64,64]” \

–primer_melody=”[60, -2, 60, -2, 67, -2, 67, -2]

Tubeman is happy! -live web final project

Tubeman is happy!


Project Overview

Tubeman always moves in an weird yet entertaining way. It shares some strong and magical happiness through its movements.

Tubeman is happy! is a tubeman-themed gif booth that is made of an interactive web application, a camera and a background board. The users can use it to create, download and share gifs of themselves moving like a tubeman.

By encouraging the users to imitate the tubeman and enabling them to create and share gifs of themselves moving, Tubeman is happy!is trying to make people happy while allowing them to share that happiness to others.


Project Setup


User Journey Map

The user will first see and go to the gif booth. They will then look at the instruction computer and follow a series instructions which basically guide them to look at a frame by frame breakdown of a tubemans movements, take pictures one frame by one frame imitating the tubeman and at the end, get a gif made of the pictures taken previously. They will be able to download and share the gifs.

All the previous taken gifs, if allowed by the users, will all be displayed on a display page that everyone can see.


Tool Kit

Web Design: paper&pen, Adobe illustrator, Adobe XD

Web Development-Javascript, HTML, CSS, node.js

Making Gif: WebRTC and gifshot.js library

Displaying, downloading and sharing gifs: nedb, web-socket

*I finished the web development and making gif part and need to work more on: 

1. designing the whole web page

2. working on enabling the users to download and share their gifs using their socket ids.


Current Demo


Working Progress

  • Design the tubeman avatar and make the gif of it moving

  • Writing the taking phones via web cam and creating gifs function. Demo:

  • Enabling Text adding function to the gifs – video

  • Adding sample movement images of the tubeman on the interface

 

 

Question left to think about

How do I make the starting taking picture action more natural and user-friendly?-possible ideas: add instruction demo videos, add a physical button to take pictures

How do I make the user to follow the movement of the tubeman in an natural way? How to guide them better? —possible ideas: add notes, add countdown timer between each photo, add guidelines on the web camera video(so they know where the arms are supposed to be)

How do I enable the users to download gifs and share them to social media like Instagram.

Where should I put my gif booth?(considering light condition…)


Some gifs ❤

Week 5 – Midterm Idea

  1. Group chat room (can enlarge/smaller onces video / can move the video around on the canvas )
  2. collective music map

(each person have a small figure that represents himself/herself when login onto this page, and he/she is able to move himself anywhere on a keyboard. And collectively all users can create a song.)

「midi controller illustration」的圖片搜尋結果

https://musiclab.chromeexperiments.com/Song-Maker/

3. Shake the balloon(tube?) man

Week 2 – Live Web – ChatBox(WebSocket)

For this week, we learned how to use web socket and server to host our website.

Installing

First, go to Digital Ocean to register an account. On Digital Ocean, we will be able to get an IP address.

Secondly, open the terminal. In the command line, follow these steps:

ssh root@the IP address from digital ocean

installing node and nmp:

apt-get update apt-get install nodejs apt-get install nodejs-legacy apt-get install npm

after those steps, create the javascript and html file
server.js/index.html –>put them somewhere on your computer

Uploading

In order to upload to your server, we need to use a tool that will allow us to connect via SSH to transfer the files. I have found that Fetch and Cyberduck are good tools.To connect, choose “SFTP”, under “Hostname” or “Server” use the IP address emailed to you by Digital Ocean. The “username” is “root” and the “password” is what you just created in Terminal.

To Run it

ssh root@YOUR_IP
cd directory_you_just_created
npm install socket.io
node server.js

website: http://YOUR_IP:8080/index.html

I just modified the sample code a bit and made it also reply to the sender's side a bit:




Week1 – Live Web -Portrait

http://206.81.13.240:8080/index.html

 

Live Platform: BiliBili (Live Bullet Comments)

 

BiliBili is a video sharing website themed around animation, comic, and game (ACG) based in China, where users can submit, view, and add commentary subtitles on videos. Bilibili uses an Adobe Flash or HTML5 player, which can be manually switched, to play user submitted videos hosted by either itself or third-party sources, while featuring a real-time overlaying subtitle system for interactive playback experience.

The interesting thing about the “bullet comment” is that while you are watching certain videos, the comments you see flying by on the screen can be either “real-time”(someone just commented while watching together with you) or it can be something that has happened before. You are basically watching a video with comments which is complied of real-time feedback and past reviews. Sometimes the significance of watching certain videos has shifted from the content of that video to a discourse emerged from it. On Bilibili, people even watch videos not really for the videos themselves but for the comments that are on the video.

People often communicate about certain topics. And platforms like BiliBili just bring the obvious topics up right in front of everyone and invite people to have a discussion. More importantly, the anonymous feature of this particular interaction probably has only made the whole progress much more engaging.