Recovery of fragmented 3GP and MP4 files

CnW Recovery software can recover many fragmented video files - a feature that very few data recovery packages even try.  Many packages just use data carving which finds sequential files, but can not recreate a fragmented file.

Cell phones and mobile phones often produce video files that are based on the Quick time format, with extensions such as .3GP, .MP4, .MOV.  As with any file, it is often possible for these file to be fragmented, and so if recovered with data carving, the file will not be valid.  CnW defragmenting routine is an option within data carving - at the end of carving the files may be optionally defragmented.  This option is part of even the most basic CnW package - because it is very important for successful recovery procedures.

Typical data structure

The file structure is a series of 4 byte lengths followed by 4 byte tags

00000000   00 00 00 18 66 74 79 70 - 33 67 70 35 00 00 03 00        ftyp3gp5

This type of file can be recognised by it’s header which started with a 4 byte length (normally 0x14, 0x18, 0x1c 0x20 or 0x24) followed by the string ‘ftyp’.  The next 4 bytes gives the file type, such as 3gp, or mp42.  There are about 30 recognised file types - CnW software will convert most to be MP4.  See here for more details www.ftyps.com

The next tag is often a ‘moov’ tag which contains all data pointers etc.  The structure is moderately complex, and can allow for multiple tracks, typically one for video, and the other for audio

00000010   6D 70 34 32 6D 70 34 31 - 00 00 0B E6 6D 6F 6F 76    mp42mp41   æmoov

The final important tag is ‘mdat’ that contains all the video and sound data.

To reconstruct the files, the process is to analyze the moov segment and determine where each video frame starts.  (This is based on information in the stco and stsz tags).  On may files, the start of the video frame is fixed data, and so data cluster can be searched for and tested to make sure they contain a frame start (or starts) at a known offset within the cluster.

One significant problem with the above description is that many files are structured  ftyp-moov-mdat, but other ones are ftyp-mdat-moov. In the later case, it is necessary to search for the matching moov fragment before reconstructing the mdat segment.  With file fragmentation, the mdat is the one most likely to be fragmented as the moov segment is often less than 64K in length.  V3.29 will process both  ftyp-moov-mdat  and ftyp-mdat-moov files up to a maximum file size of 16MB.  Later versions will support any file length..  Current success rate is between 0 and 100%, it all depends how fragmented the files are, and if the fragments do actually exist.  We have also seen files where there is an incomplete mdat segment and no moov segment at all.  Also, the length of the ftyp header is zero  A future development plan is to reconstruct the moov segment

How defragmenting works

The basic structure of the file is one of a four byte length, then a four byte tag, followed by data.  This means that the location of the next tag is always known, although the type of tag can only be guessed.  For the mdat segment, unfortunately there is only a single header and tag, but the moov segment may have a 100 or so tags.  Typically the moov segment is short, but a complete one is essential to view a possibly incomplete mdat.

With defragmenting, it is assumed that data is always in original clusters, although the location of a cluster is not known.  This mens that if a tag is expected at location 0x579421 in a file, and the cluster size is 0x4000, then clusters are tested to see if they have a suitable tag in location 0x1421  (ie 0x579421 modulus 0x4000).  For many moov segments this will produce a good reliable result.  Problems arise on long files, and sometimes the tag ‘stts’ and ‘stsz’ can be very long. If this sits over a fragment location, then different functions are required to analyse data and try and add the relevant pieces.

To reconstruct mdat segments is it largely essential to have a valid moov segment.  From this, key elements can be found, eg frame starts, and matched with possible data clusters.

No moov segment

Video has been come across where the camera did not complete the file.  ie it has a dummy ftyp header, a mdat section but no moov.  The indication that this is an incomplete file, rather than just fragmented is that the ftyp header length is set as zero.  CnW is currently working on creating a moov section to enable the mdat to be displayed.

Non sequential recording

An issue with several cameras is that the data is not recorded onto the memory chip sequentially.  This is because the moov segment can only be created when the mdat segment is complete.  The approach is to record the segements as ftyp-mdat-moov with each segment on a cluster boundary.  The FAT on the disk can then be changed to that logically the data is ftyp-moov-mdat.  However, when files are deleted, this modified FAT is lost and recovery difficult.  Fortunately, CnW Recovery defragmentation software detects this pattern and corrects the files, allowing for very successful recovery.

User support

CnW is very keen to help ever customer.  If a situation arises where recover has not been possible, CnW will assist.  If necessary, the media, or a copy, can be sent to CnW who will extract the files.  As this is both good customer support, and useful development, there is no extra charge for this work - except possible postage costs.

3GP is actually a simplified version of MPEG-4 (MP4).  Video frames can either be MPEG-4 or H.263

Click here for a white paper on the subject.