Multimedia, compression standards and animation notes — Unit 5

Multimedia is the seamless integration of multiple forms of media (text, graphics, audio, animation, and video) into a single, cohesive computerized system.

⇒1.1 Key Characteristics

• Digitization: All media types are converted into a common digital format (binary code), allowing them to be manipulated by a computer.
• Interactivity: Unlike a traditional TV broadcast, multimedia systems allow the user to control the flow, timing, and sequence of information.
• Synchronization: Time-based media (audio and video) must be perfectly synchronized during playback.

⇒2.1 Static Media

• Text: The foundation of information delivery. Can be formatted (Rich Text) or unformatted.
• Graphics: Still images, illustrations, and diagrams. Crucial for visual communication.

⇒2.2 Continuous (Time-Based) Media

• Audio: Digitized sound waves (speech, music, sound effects).
• Animation: Rapid display of a sequence of static images (2D or 3D) to create the illusion of motion.
• Video: Capturing, recording, and playing back moving images, typically accompanied by audio.

Raw multimedia files are astronomically large. Uncompressed digital audio (CD quality) requires about 10 MB per minute. Uncompressed 1080p video at 60fps requires nearly 20 GB per minute.

⇒3.1 Storage and Bandwidth

Without compression, it would be impossible to store movies on DVDs, or stream YouTube videos over standard internet connections. Compression algorithms reduce file sizes by finding and eliminating redundancies in the data.

Lossless compression reduces file size without losing a single bit of information. When the file is decompressed, the resulting data is 100% identical to the original.

⇒4.1 Applications

Mandatory for text documents, executable programs, and source code, where losing even one character could destroy the file. (Examples: ZIP files, PNG images, FLAC audio).

⇒4.2 Common Algorithms

• Run-Length Encoding (RLE): Replaces repeating data with a single value and a count (e.g., AAAAA becomes 5A).
• Huffman Coding: Uses variable-length codes based on frequency. Common characters get short codes (e.g., 2 bits), rare characters get long codes.

Lossy compression permanently discards non-essential information to achieve massive file size reductions (e.g., 90% smaller). The decompressed file is an approximation of the original.

⇒5.1 Psychoacoustics and Human Vision

These algorithms exploit the limitations of human perception.

• Vision: The human eye is very sensitive to brightness, but terrible at detecting subtle changes in color (chrominance).
• Hearing: If a very loud sound plays, the human ear cannot hear a quiet sound played at the same exact time (auditory masking).

Lossy algorithms mathematically identify this 'invisible' or 'inaudible' data and simply delete it. (Examples: JPEG, MP3, MP4).

The Joint Photographic Experts Group (JPEG) created the standard for lossy compression of continuous-tone still images.

⇒6.1 Step 1: Color Space Conversion

The image is converted from RGB to YCbCr. 'Y' represents Luminance (brightness). 'Cb' and 'Cr' represent Chrominance (color).

⇒6.2 Step 2: Chroma Subsampling

Because human eyes are bad at seeing color detail, the algorithm immediately throws away half (or more) of the Cb and Cr data, while keeping all the Y data.

⇒7.1 Step 3: Discrete Cosine Transform (DCT)

The image is broken into 8x8 blocks. The DCT mathematically translates the spatial pixel data into a frequency domain. It separates the block into low-frequency data (gradual color changes) and high-frequency data (sharp edges and fine textures).

⇒7.2 Step 4: Quantization

The crucial 'lossy' step. A mathematical matrix divides the DCT frequencies. The high-frequency numbers become so small they are rounded down to exactly 0. The fine details are permanently erased based on the 'Quality' slider chosen by the user.

Step 5 is Entropy Coding, which losslessly zips up all those zeros using Huffman coding.

The Moving Picture Experts Group defines standards for video compression (like MPEG-2 for DVDs, and MPEG-4/H.264 for streaming).

⇒8.1 Spatial vs Temporal Redundancy

A video is just 30 or 60 JPEG images played every second. You can compress each frame individually using JPEG techniques (Spatial Redundancy).

However, the real power of MPEG comes from Temporal Redundancy. In a video of a news anchor, the background doesn't move. Frame 1 and Frame 2 are 99% identical. It is wildly inefficient to store the background twice.

MPEG achieves compression by creating a Group of Pictures (GOP) consisting of three frame types.

• I-Frame (Intra-coded): A complete, standalone image, compressed like a JPEG. It serves as a reference point. Lowest compression.
• P-Frame (Predictive): It only stores the changes from the preceding I-Frame or P-Frame. It uses Motion Vectors to say 'Move this block of pixels 5 units to the right'. High compression.
• B-Frame (Bi-predictive): It analyzes both the past frame and the future frame to calculate motion vectors. Highest compression, but requires the most processing power to decode.

Animation is the technique of photographing successive drawings or models to create an illusion of movement when the movie is shown as a sequence.

⇒10.1 Frame Rates

The illusion relies on the persistence of vision. Classical film runs at 24 frames per second (fps). Video games and smooth computer animations typically target 60 fps to reduce motion blur and input latency.

Historically, lead animators at Disney would draw the 'Key' frames (e.g., the exact moment a character jumps, and the exact moment they land). Junior animators would then painstakingly draw all the 'In-between' frames.

⇒11.1 Computer Tweening

In computer animation, the software acts as the junior animator. The user defines the keyframes (e.g., setting a 3D car's X-position to 0 at frame 1, and 100 at frame 60). The software interpolates (tweens) the values to automatically generate frames 2 through 59.

Animating complex character skeletons (like a human arm with a shoulder, elbow, and wrist joint) requires specialized mathematics.

⇒12.1 Forward vs Inverse Kinematics

• Forward Kinematics (FK): The animator rotates the shoulder, which moves the elbow. They then rotate the elbow, which moves the wrist. The motion flows strictly down the hierarchy. Good for swinging motions.
• Inverse Kinematics (IK): The animator places the hand exactly on a door handle. The software calculates the reverse math to automatically bend the elbow and shoulder into the correct realistic angles to reach that point. Essential for feet touching the ground without slipping.

A special effect in motion pictures and animations that changes (or morphs) one image or shape into another through a seamless transition.

⇒13.1 How it works

It is more complex than a simple cross-fade (where image A dissolves into image B). Morphing involves simultaneous warping and fading.

• An animator maps key points on the source image to corresponding points on the target image (e.g., mapping the eyes of a man to the eyes of a tiger).
• The software geometrically warps the shape of image A towards image B while simultaneously fading the colors.

The process of recording the movement of objects or people.

⇒14.1 Application in 3D Animation

Instead of a human animator manually keyframing a character's walk cycle, an actor wears a suit covered in reflective markers. Specialized cameras track these markers in 3D space. This data is then applied directly to a digital skeleton, yielding perfectly realistic, human-like animation in a fraction of the time.