The .ser.mj2 format

Standard ISO base media / JP2 family boxes:

jP      12 bytes    signature
ftyp    20 bytes    brand 'mjp2'
serR    (v4 only)   copy of the metadata, see below
mdat                the frames
moov                index: stsd / stts / stsc / stsz / stco or co64
serM                the metadata

mdat holds each stored image as its own jp2c box: a 4-byte big-endian length, the tag jp2c, then a raw JPEG 2000 codestream starting FF 4F FF 51. stsz records the sample length including that 8-byte header, and stco/co64 point at the header rather than at the codestream. That is the ordinary MJ2 convention, so a stock decoder agrees without being told anything.

Two consequences worth knowing:

• The samples form a self-delimiting chain. Each says how long it is, so they can be walked from the first one without moov at all. That is how a truncated file is recovered.
• Files over 4 GB use co64. A lunar capture reaches that easily.

Encoding is reversible 5/3, lossless: 64×64 code-blocks, 5 decomposition levels, LRCP, one layer, no multi-component transform (except layout 5, which does use the reversible color transform).

planesPerFrame consecutive stored images make one SER frame. A decoded image is height rows × width columns in normal raster order, and the SER payload is those same pixels row-major — so no transpose is needed; flatten each image row by row.

Layout 0 — the samples are the pixels. Done.

Layout 4 — the common 16-bit case, and the only one needing arithmetic:

sample = (quotient << shiftK) + remainder + offsetMin

with the quotient the first plane and the remainder the second. shiftK is 4 and bitDepth 12 in every file written so far, but read both from the box rather than assuming them.

A 12-bit sensor written into 16 bits leaves the remainder plane constant, which is why this layout is chosen for it. A reader never has to make that choice — layout says what was done. Appendix A has the rule for anyone writing files.

Layout 2 and 3 — sample = (hi << 8) | lo, per channel for layout 3, plane order R.hi R.lo G.hi G.lo B.hi B.lo.

Layout 1 — three gray planes per frame, interleave as R G B per pixel.

Layout 5 — one 3-component image; interleave the components per pixel. The image is always stored in R,G,B order so ordinary players show it correctly, but SER ColorID 101 means the source was B,G,R — reverse the channel order on the way out for that ColorID, or the colors come back swapped.

For a mono or Bayer capture — the overwhelmingly common case — a reader is:

1. open the file with any MJ2 decoder
2. read the last 12 bytes, seek back, parse serM
3. write out the 178 header bytes
4. for each frame, decode it and write the pixels row by row
5. write out the trailer bytes

Layout 4 adds one line of arithmetic to step 4. Everything else on this page is the less common cases.

The JPEG 2000 decoding is your library's job. What follows is only the part specific to this format: finding the box, reading its fields, and turning decoded planes back into SER samples. Error handling is thin, for brevity — a real reader should check every length against the file size.

#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>

/* 64-bit seek, whatever the platform calls it.  See the warning above. */
#if defined(_WIN32)
  #define serz_seek(f, off, whence)  _fseeki64((f), (int64_t)(off), (whence))
#else
  #define _FILE_OFFSET_BITS 64
  #define serz_seek(f, off, whence)  fseeko((f), (off_t)(off), (whence))
#endif

/* Fields are little-endian inside the box, whatever the host is. */
static uint32_t rd32(const uint8_t *p){
    return (uint32_t)p[0] | ((uint32_t)p[1]<<8) |
           ((uint32_t)p[2]<<16) | ((uint32_t)p[3]<<24);
}
static uint64_t rd64(const uint8_t *p){
    return (uint64_t)rd32(p) | ((uint64_t)rd32(p+4) << 32);
}

typedef struct {
    uint32_t version, layout, bitDepth, planes;
    uint64_t origBytes;
    uint32_t headerLen, trailerLen, offsetMin, shiftK, nHash;
    uint8_t  header[178];
    uint8_t *trailer;          /* trailerLen bytes  */
    uint8_t *digests;          /* 16 * nHash bytes  */
} serz_meta;

int serz_read_meta(FILE *f, serz_meta *m)
{
    uint8_t tail[12];
    if (serz_seek(f, -12, SEEK_END) != 0)       return -1;
    if (fread(tail, 1, 12, f) != 12)            return -1;
    if (memcmp(tail + 4, "SERZEND1", 8) != 0)   return -1;

    uint32_t boxSize = rd32(tail);              /* the locator is LE */
    /* Backwards from the end, so no file length is needed and nothing can
     * overflow.  Do NOT compute (fileLength - boxSize) with ftell. */
    if (serz_seek(f, -(int64_t)boxSize, SEEK_END) != 0) return -1;

    uint8_t *b = malloc(boxSize);
    if (fread(b, 1, boxSize, f) != boxSize)   { free(b); return -1; }
    if (memcmp(b + 4, "serM", 4) != 0 ||
        memcmp(b + 8, "SERZ", 4) != 0)        { free(b); return -1; }

    m->version    = rd32(b + 12);
    m->layout     = rd32(b + 16);
    m->bitDepth   = rd32(b + 20);
    m->planes     = rd32(b + 24);
    m->origBytes  = rd64(b + 28);
    m->headerLen  = rd32(b + 36);
    m->trailerLen = rd32(b + 40);

    m->offsetMin = m->shiftK = m->nHash = 0;
    size_t at = 44;                              /* v1 stops here */
    if (m->version >= 2) { m->offsetMin = rd32(b + 44);
                           m->shiftK    = rd32(b + 48); at = 52; }
    if (m->version >= 3) { m->nHash     = rd32(b + 52); at = 56; }
    if (m->version >= 4) { at = 88; }            /* two 16-byte digests */

    memcpy(m->header, b + at, 178);
    at += m->headerLen;
    m->trailer = malloc(m->trailerLen);
    memcpy(m->trailer, b + at, m->trailerLen);
    at += m->trailerLen;
    if (m->nHash) {
        m->digests = malloc(16u * m->nHash);
        memcpy(m->digests, b + at, 16u * m->nHash);
    } else {
        m->digests = NULL;
    }
    free(b);
    return 0;
}

serR at the front of a v4 file is the same body without the trailing locator, so the same field offsets apply once you have found the box by walking jP, ftyp, … from offset 0.

Turning decoded planes into SER samples, for the two layouts still written for mono and Bayer data:

/* layout 0 - the samples are the pixels */
memcpy(out, plane[0], npx * bytesPerSample);

/* layout 4 - quotient then remainder */
for (size_t i = 0; i < npx; i++)
    out[i] = (uint16_t)((quotient[i] << shiftK) + remainder[i] + offsetMin);

And the whole file is then:

write(header, 178)
for each frame:  decode its planes, transform, write npx samples
write(trailer, trailerLen)

which reproduces the original .ser byte for byte. If nHash is non-zero, the MD5 of each frame's written bytes should equal the corresponding 16 bytes of digests — that is the check that a decoder is exact, and it needs no original to compare against.