-
- Downloads
Added MPI_Barrier to split file reader
This update adds an explicit MPI Barrier to the end of the split 2D file reader. This should address a race condition discovered by Nancy when resuming a 2D case as 3D on Scinet, where the initially-2D files were (correctly) overwritten by their 3D versions. It appears that the overwrite happened on some processes before others had opened the file, causing the later processes to read incomplete files. The exact sequence appears to have been: * Process 1 opens 'u.0' as a 2D-split file to read * Process 1 finishes its reading and assigns the contents to its internally-3D array * Process 1 begins writing out its segment of the 3D array, as 'u.0'. (It blocks on the write-synchronization, but has already written to disk.) * Process 2 wakes up (on a remote node) and opens 'u.0' as a presumably-2D file. It dies. This race condition could, under different circumstances, also cause data corruption (if process N wasn't reading past the end of the file in a linear sense; possible for large Ny and small Np). The corrective measure was to apply an MPI Barrier at the end of read_2d_slice in Split_reader_impl.cc (the implementation fucntion used by the more user-friendly functions in Science.cpp).
Loading
Please register or sign in to comment