.vtu/.vtp output >4 GB unreadable — 64-bit offsets needed - Potential Fix for MFIX-26*

MFiX
1

1. The issue.

On large cartesian-grid runs, .vtu/.vtp files became unreadable in ParaView once they crossed ~2 GB. The appended-data byte offsets (offset="…") and the per-block size prefixes were default 4-byte INTEGER, so they overflow at 2³¹. The files still wrote to full size, but the offset values wrapped to garbage and ParaView could no longer parse them.

2. Key modifications

(3 files: cartesian_grid/vtk_mod.f, cartesian_grid/vtk_out.f, des/vtp_mod.f):

  • VTU_offset and the per-routine nbytes_* / offset_* locals → INTEGER(KIND=8).

  • Byte-size products evaluated in 64-bit via INT(count,8), so the count×components term can’t overflow before promotion.

  • New module parameter VTK_HEADER_SIZE = 8: the appended-block size prefix grew 4→8 bytes, so every offset = prev + <prefix> + nbytes term now advances by 8 — not just the variable kinds (the bookkeeping point flagged on the channel).

  • header_type="UInt64" added to the <VTKFile> tags that carry appended data (UnstructuredGrid .vtu, PolyData .vtp); offset output field widened I12 → I20.

  • Used KIND=8 / c_int64_t throughout (8 bytes on both LP64 and LLP64), never c_long, so the 8-byte prefixes stay consistent with the existing byte_order="BigEndian", ACCESS='STREAM' writes on Windows and Linux. (One local literally named int was renamed int4 — it shadowed the INT() intrinsic.)

Payloads are unchanged (still Float32/Int32); files grow 4 bytes per appended block. Validated on HPC >3 GB .vtu files now write and open correctly in ParaView (When the default MFIX-25.3 or MFIX-26.1.1 did not). The fix was tested with some of the 2D TFM and DEM-CFD tutorials. Note Paraview 6.1.0 cannot open the files but Paraview 6.0* can :slight_smile: . The modified three subroutines are attached in the zip file.

vtk_mod.zip (48.3 KB)

1 Like
2

Thanks Eric. We’ll do some testing and probably include your 64-bit patch in the next MFiX release (unscheduled at this time)

Re: the Paraview 6.1 issue:
Paraview doesn’t like the space character in front of the timestep value in the pvd file, they already have a fix for this (thanks for reporting the issue, it’s nice to see the open-source ecosystem working as intended):

vtkPVDReader: Fix parsing of values (!7827) · Merge requests · ParaView / ParaView · GitLab

We can also avoid creating the blank space on the MFiX side:

pvd_patch.txt (2.6 KB)

This will also be included in the next MFiX release.

– Charles

1 Like