The Sound of Presence

Introduction

This is music-driven animation inspired by a preliminary idea created by Zhengyang. That first concept included a realistic visual style in which a small ball moves across a revolving staircase, moving with the music. However, early group conversations revealed numerous drawbacks to this model, notably with visual clarity and audience engagement. We were striving to integrate audio-visual concepts so that our project could be seen by a large audience (students as well as ordinary commuters like members of public transport). To align it more with this cohort, we focused on a more stylised, cartoon-type style of design; making the aesthetic more lighthearted and soothing. The final result is offered through video with fusion of 3D animation and 2D visuals.

Concept development

I was at the beginning and working on my (about Week 4) stage of the project, so I was really instrumental in concept evaluation and refinement. Two important problems that emerged with the original design. The first reason is that there is a visually cluttered realistic background, making the viewer lose focus from the main subject. Second, a ball rolling down a staircase was also too repetitive and unappealing to the eye. Given this analysis, I suggested a new direction to retain this essential principle of “movement driven by music” but to rework the visual and interaction design. With the new concept, for every bounce of the ball is set into rhythm to the music to dynamically generate keys for the piano. I also suggested to redesign the staircase into an integrated series of musical elements:

the steps are piano keys.
the handrails symbolize the strings of the cello.
the vertical bars are similar to guitar strings.
the details of the pillar represent the drum beats.

This change transformed the environment into a visualisation of music, facilitating the link (via image) between sound and imagery.

My contribution

i am the team leader and the manager. I was also mainly responsible for music production in FL Studio during this project. Since the only person from the team that had both background in music and composition skills, I was solely tasked with creating the complete soundtrack. My task had a deadline of 2 weeks, but I managed to complete the piece within as little as 10 days. The music is the foundation of this project and all animation revolved around it so I knew to finish as early as possible to support the work in the pipeline. I had wanted a laid-back, paced tempo appropriate for animation of 90 BPM. My choice of the instrument was limited to four, drums, piano, cello & guitar. Instead of advanced mixing methods, I used clear and straightforward audio layers that would enable the animation team to easily sync visual elements with certain sound cues. Besides music production, I had a major impact on:

concept development & ideation refinement.
coordinating and tracking group progress.
analysing team concepts.
some technical help.
lighting and rendering.

In general my participation was spread over a variety from the start of the project up to the end time.

Process & Problem Solving

I also helped the team to solve technical issues with the production process from start to finish. Like for the animated guitar strings, team-mates were struggling producing realistic movement. Then I suggested the concept of using soft selection tool in Maya to make it smoother, more natural deformation. I suggested for the drum animation that the team follow the peaks of the waveform of the audio. As drum beats dominate the waveform, this method offered accurate timing with no additional audio processing. Creating the staircase handrail’s “growth” effect proved to be also a major challenge, and was inspired by the cello structures. I suggested ‘Boolean’ tool to smooth out the steady view of geometry. Also, we resolved to find a solution with the tutor and spoke to the team, and it worked. Our weekly in-person meetings with regular group discussions helped us identify problems and solve them together as a group. This efficiency was also enhanced in the workflow itself.

For the lighting stage, my choice here was gentle soft lighting. In the final composite, this was selected in order to be compatible with the 2D cartoon-style background. However, this test version has following disadvantages:

The style of the 3D result is not suitable for 2D background
The bouncing ball seems not clear enough to engage the audience
The bouncing animation is too boring

Process & Problem Solving

For camera design, I made a motion path based on a duplicated curve formed from the staircase handrail. The camera follows this track via a camera and aim setup and remains focused on the main subject, producing a fluid and controlled visual experience.

We have maintained a more realistic aspect of the scene so throughout Week 7 we did the first stage. But this wasn’t a visual match for the 2D backdrop, which gave me a visual inconsistency. Then I brought Arnold AI Toon Shader in to fully stylise the scene for the sake of resolving it. First, I tried AOV rendering (Arbitrary Output Variables), but these methods were inefficient and didn't give me enough room for compositing. So I went with an entirely stylised form of shading. I used cold colours and consciously avoided shades of yellow as the background was just soft yellow. This prevented the foreground elements from fading into the background. I also added gradient-based colour variation throughout everything to make the animation vibrant and visually appealing. Final rendering settings set:

Contour filter (value 8) for outlines
Gaussian filter (value 2) for shading
PNG format with Alpha channel for use in After Effects compositing

Result Achieving

The end result is a striking animation where sound is at the helm of both movement and environment. The final result of the work is much more stylised, fanciful, appealing, and conceptual. In the last stage I made a suggestion, to take inspiration from the “Bad Apple!!” music video. By using line-art render passes since I had kept them, I advocated mixing between coloured cartoon graphics and black-and-white line-art animation. This idea was well received by the team at first, as well as introducing visual contrast and variety into the finished product.

However, after a discussion with our tutor and reviewing the whole video again and again, we finally decided to delete the black-and-white line-art animation. Also, we need to refine the background elements and the animation.

Finally, the final render sequence was completed on time.

Reflection and Conclusion

The project went well finally. Most were technical and quickly dealt with by discussion and collaboration. The team members were willing to take the wheel and articulate their thoughts and doubts, so it was a really great workflow. But I noticed a weakness in our project management. As one of the lead project organisers I failed to create a transparent file naming system at the starting point of the project. The end result was some inefficiencies later on, with the team struggling to find files and keep track of them. Next time, this would be another area in which I could improve.

However, team communication and teamwork were evident. This proved especially fruitful in weekly in-person discussion, able to address problems in ways easier than online. We also got some useful feedback from tutors and peers. This project has massively contributed to improving my music production, rendering, and animation pipeline.

More importantly, it sharpened my skills in planning, directing, and managing a team. I think the end result is interesting and creative, and the process has really opened my eyes for other collaborative projects. My future goal is always to grow technologically, yet developing some aspect about leadership to further evolve into a technical director or creative lead in the animation and visual effects space.