This module provides parallel acces to netCDF files. One netCDF dimension is defined for splitting the data among processors such that each processor is responsible for one slice of the file along that dimension.
Since netCDF files can be very big, the distribution algorithm gives priority to memory efficiency over CPU time efficiency. The processor that handles the file treats only one slice per superstep, which means that at no time more than one slice must be stored in any processor.
Constructor: ParNetCDFFile(filename, split_dimension, mode=r, local_access = 0)
the name of the netCDF file
the name of the dimension along which the data is distributed over the processors
read (r), write (w), or append (a). Default is r.
if 0 (default), processor 0 is the only one to access the file, all others communicate with processor 0. If 1 (only for reading), each processor accesses the file directly. In the latter case, the file must be accessible on all processors under the same name. A third mode is auto, which uses some heuristics to decide if the file is accessible everywhere: it checks for existence of the file, then compares the size on all processors, and finally verifies that the same variables exist everywhere, with identical names, types, and sizes.
A ParNetCDFFile object acts as much as possible like a NetCDFFile object. Variables become ParNetCDFVariable objects, which behave like distributed sequences. Variables that use the dimension named by split_dimension are automatically distributed among the processors such that each treats only one slice of the whole file.