AtomicParsley
Links

AtomicParsley is a lightweight command line program for reading, parsing and setting metadata into MPEG-4 files supporting these styles of metadata:

  • iTunes-style metadata into .mp4, .m4a, .m4p, .m4v, .m4b files
  • 3gp-style assets (3GPP TS 26.444 version 6.4.0 Release 6 specification conforming) in 3GPP, 3GPP2, MobileMP4 & derivatives
  • ISO copyright notices at movie & track level for MPEG-4 & derivative files
  • uuid private user extension text & file embedding for MPEG-4 & derivative files

The MPEG-4 container is based on Apple's Quicktime .mov container (just more sane) & there are a number of similarites - as well as differences. Each MPEG-4 file lists a "major file brand" that broadly says which specification the file adheres to. For example, an iTunes audio file typically lists 'M4A ' as its major file brand, while a 3GPP2 .3g2 file has something like '3g2a'. It is based on this major file brand that AtomicParsley allows a style of metadata to be set. Since there is nothing in the 3GPP2 specification regarding iTunes-style metadata, its use is prevented on such files. Similarly, a 3gp 'location' asset is not available for an iTMS video file. The first paragraph here sums the situation up well.

With the exception of artwork, all iTunes-style tags support only 1 piece of metadata - you can't have for example 3 title tags. 3gp assets and copyright notices support multiple tags, but they must differ in the language setting for that tag. You can have for example 3 title tags, but 1 in english, 1 in spanish, 1 in undefined. All iTunes-style text metadata is always in UTF-8; 3gp assets & copyright notices can be in set in UTF-8 or UTF-16. All strings are converted internally to UTF-8, converting as necessary. On *nix platforms, input is in UTF-8; the native Windows port supports full UTF-16 input.

_____________

The MPEG-4 specification details the MPEG-4 container. It is comprised of discrete units called 'boxes' or 'atoms'. An atom is defined as being a very simple building block:

[4bytes atom length] [4bytes atom name] [contents of the atom, if any]

Just as a gold or carbon atom have particular characteristics, each atom name has specific characteristics. For example a moov atom is a parent atom with no content except other atoms. mdat atoms carry the raw audio/visual stream & so have lots of content - typically the largest portion of the file. If a file is not written properly or is non-conforming, the file will either not play or become untouchable by other mpeg-4 utilities.

AtomicParsley works by first scanning the file, looking for these atoms and how they are organized. Here is an example of a tree (here with track level details) for an h.264 avc file:

 

Atom ftyp @ 0 of size: 32, ends @ 32

Atom moov @ 32 of size: 23276, ends @ 23308

Atom mvhd @ 40 of size: 108, ends @ 148

Atom trak @ 148 of size: 3664, ends @ 3812

Atom tkhd @ 156 of size: 92, ends @ 248

Atom edts @ 248 of size: 36, ends @ 284

Atom elst @ 256 of size: 28, ends @ 284

Atom mdia @ 284 of size: 3484, ends @ 3768

Atom mdhd @ 292 of size: 32, ends @ 324

Atom hdlr @ 324 of size: 58, ends @ 382

Atom minf @ 382 of size: 3386, ends @ 3768

Atom vmhd @ 390 of size: 20, ends @ 410

Atom dinf @ 410 of size: 36, ends @ 446

Atom dref @ 418 of size: 28, ends @ 446

Atom stbl @ 446 of size: 3322, ends @ 3768

Atom stsd @ 454 of size: 176, ends @ 630

Atom avc1 @ 470 of size: 160, ends @ 630

Atom avcC @ 556 of size: 46, ends @ 60

Atom uuid=6b6840f2-5f24-4fc5-ba39-a51bcf0323f3 @ 602

Atom stts @ 630 of size: 24, ends @ 654

Atom stss @ 654 of size: 28, ends @ 682

Atom sdtp @ 682 of size: 434, ends @ 1116

Atom stsc @ 1116 of size: 700, ends @ 1816

Atom stsz @ 1816 of size: 1708, ends @ 3524

Atom stco @ 3524 of size: 244, ends @ 3768

Atom udta @ 3768 of size: 44, ends @ 3812

Atom cprt [eng] @ 3776 of size: 36, ends @ 3812

Atom trak @ 3812 of size: 3873, ends @ 7685

Atom tkhd @ 3820 of size: 92, ends @ 3912

Atom edts @ 3912 of size: 36, ends @ 3948

Atom elst @ 3920 of size: 28, ends @ 3948

Atom mdia @ 3948 of size: 3737, ends @ 7685

Atom mdhd @ 3956 of size: 32, ends @ 3988

Atom hdlr @ 3988 of size: 58, ends @ 4046

Atom minf @ 4046 of size: 3639, ends @ 7685

Atom smhd @ 4054 of size: 16, ends @ 4070

Atom dinf @ 4070 of size: 36, ends @ 4106

Atom dref @ 4078 of size: 28, ends @ 4106

Atom stbl @ 4106 of size: 3579, ends @ 7685

Atom stsd @ 4114 of size: 103, ends @ 4217

Atom mp4a @ 4130 of size: 87, ends @ 4217

Atom esds @ 4166 of size: 51, ends @ 4217

Atom stts @ 4217 of size: 24, ends @ 4241

Atom stsc @ 4241 of size: 784, ends @ 5025

Atom stsz @ 5025 of size: 2388, ends @ 7413

Atom stco @ 7413 of size: 272, ends @ 7685

Atom udta @ 7685 of size: 15623, ends @ 23308

Atom cprt [eng] @ 7693 of size: 24, ends @ 7717

Atom meta @ 7717 of size: 15591, ends @ 23308

Atom hdlr @ 7729 of size: 34, ends @ 7763

Atom ilst @ 7763 of size: 15545, ends @ 23308

Atom covr @ 7771 of size: 15454, ends @ 23225

Atom data @ 7779 of size: 5156, ends @ 12935

Atom data @ 12935 of size: 10290, ends @ 23225

Atom Šnam @ 23225 of size: 43, ends @ 23268

Atom data @ 23233 of size: 35, ends @ 23268

Atom aART @ 23268 of size: 40, ends @ 23308

Atom data @ 23276 of size: 32, ends @ 23308

Atom free @ 23308 of size: 2048, ends @ 25356

Atom mdat @ 25356 of size: 2179919, ends @ 2205275

------------------------------------------------------

Total size: 2205275 bytes; 59 atoms total. AtomicParsley version: 0.9.0 (utf8)

Media data: 2179919 bytes; 25356 bytes all other atoms (1.150% atom overhead).

Total free atom space: 2048 bytes; 0.093% waste. Padding available: 2048 bytes.

------------------------------------------------------

Movie duration: 14.081 seconds (0:14.08) - 1238.52* kbp/sec bitrate (*=approximate)

Low-level details. Total tracks: 2

Trk Type Handler Kind Lang Bytes

1 vide Apple Video Media Handler avc1 eng 1962254

1114.86* kbp/s 14.081 sec AVC Baseline Profile, Level 3 640x480 (1200 macroblocks)

Creation Date (UTC): Thu Sep 14 16:12:24 2006

Modification Date (UTC): Thu Sep 14 16:12:26 2006

2 soun Apple Sound Media Handler mp4a eng 217641

128.00 kbp/s 13.746 sec MPEG-4 AAC Low Complexity Profile channels: [2]

Creation Date (UTC): Thu Sep 14 16:12:24 2006

Modification Date (UTC): Thu Sep 14 16:12:26 2006

The actual raw aac/avc streams are in the last atom: mdat. You can see here that the moov atom is populated with a number of atoms. The information that iTunes lets you set is in the moov.udta.meta.ilst hierarchy. You can see here atoms covr, Šnam, & aART which are respectively artwork, name & album artist. You'll note that under each atom is a data atom. It is this data atom that holds the actual information you see in iTunes.

Also present are copyright notices on track 1 and at movie level on the cprt atom. This atom is different from an iTunes-style metadata tag in a number of ways. It is present directly under the 'udta' atom, it does not contain a data atom, and it carries a language setting. Not shown is that it can also be set in utf8 or utf16 and there can be 1 or more differing in the language setting. While iTunes can't see these notices, Quicktime Player can - both at movie and track level, but only for the first language.

Another type of atom called the uuid form is present in track 1. This particular file was created with Apple's Compressor program for the iPod setting. The uuid form allows for user-extensible setting of arbitrary metadata.

You also see a free atom here. A free atom is purely emply NULL space. When found in a specific area in the file, it can function as padding: when you change a tag from 5 characters to 35, the free atom becomes 30 bytes smaller, and minimal alteration of the file occurs. AtomicParsley can utilize available padding when used with the --overWrite switch, but defaults to non-destructive rewriting of the entire file. Using the free space dynamically is beneficial for rapid changing of tags, but as seen in this faac example tagged by mp4tags:

------------------------------------------------------
Total size: 13923804 bytes; 45 atoms total. AtomicParsley v0.7.5b
Media data: 12459082 bytes; 1464722 bytes all other atoms (10.520% fluff).
Total free atoms: 127556 bytes; 0.916% waste.
------------------------------------------------------

That's 128 kilobytes. Over the span of dozens of files.... that's many megabytes. Enough for a few extra songs. It is possible to delete some or all free atoms using AtomicParsley (and spare any available padding). Be aware that if you alter the tags using another program (iTunes or something based on mpeg4ip's libmp4v2 like mp4tags), there will probably be a re-appearance of free atoms. In fact, due to the way that mp4tags works, if your mdat atom comes after your moov atom, mp4tags will duplicate the moov atom at the end (which means moving mdat ahead of moov), modify the tags and then NULL out the original moov into a free atom. Not terribly efficient space-wise, but that's what accounts for its speed. It also prevents the file from streaming.

Additionally, in the above tree you will see a stco atom for each track. It is the "Sample Table Chunk Offset" table, and here is what iTunes (and every other player) uses to find chunks of data in the file. If the audio data moves & your mdat atom was at the end of your file, the locations of that data needs to be adjusted as well. In some files, this becomes a 64-bit atom, and is called co64.

AtomicParsley defaults to non-destructive writing to a temp file next to the source file - make sure you have you have the room available. This will be the default behavior for some time. You can append a "--overWrite" command line flag and the original file will be deleted and the temp file will be renamed to the original filename. Also, a problem more prevalent on Windows, you may bump up to the limitation of 255 characters in the fullpath & filename - you should hear a beep.

 

Projects that use AtomicParsley

Mac OS X projects:

ID Infiltr8 - batch process mp3 & m4a audio files; set tags, rename files & internet searches (opensource) - Mac OS X ≥ 10.4

vID Infiltr8 - video metadata tagger with artwork support & untested universal binary (opensource) - Mac OS X ≥ 10.4

Lostify - mpeg-4 video metadata tagger GUI - Mac OS X ≥ 10.3.9

Parsley Is Atomicly Delicious - streamlined tagger focusing on TV Show-related tags (opensource) - Mac OS X

Cross-platform projects:

iTags.widget - Yahoo widget that supports most iTunes tags & batch setting. Discuss here or here

Windows-based projects:

TVTagger - Java based mpeg-4 tagger GUI to set tv-related tags - Windows

iPodTVShow - batch set tv-related tags into mpeg-4 video files - Windows

AtomicParsley GUI - supports most tags in mpeg-4 files including the TV show tags - Windows (discuss here)

YAMB - advanced front end to GPAC's mp4box & sets most known tags - Windows (discuss here)

WinAP - tagger geared for TV show tags; supports batch files & overwriting (soon to be opensourced) - .net 2.0 for Windows

 

SourceForge.net Logo

Links
Links
item6a1 item6a item3a item6b item3b item2a