Access to Heavyweight Data
Last update: 10.07.2023
Access to Heavyweight Data
Heavyweight data is data representing mesh, point cloud, and solution field data. Often, solvers store heavyweight data in contiguous arrays in memory, and the participant library design takes advantage of this. The participant solver provides heavyweight data access to the participant library, but retains ownership of all heavyweight data (i.e., it is responsible for allocation and deallocation of those arrays).
For more information, see:
Relevant Concepts
Input data
Input data is for input variables provided to the participant by System Coupling. The participant library needs to have read and write access to this data.
Output data
Output data is for output variables provided to System Coupling by the participant, including mesh data. The participant library needs to have read-only access to this data.
Scalar and Vector Data
Scalar data has one component. Temperature values are an example of scalar data. Vector data has the same number of components as there are dimensions, i.e. three components in 3-D cases. Nodal coordinates values are an example of vector data.
Real and Complex Data Types
System Coupling supports for real- and complex-valued variables. Complex values have real and imaginary components. Term real is usually ommitted, so unless data is specifically referred to as complex, real-valued data should be assumed.
Integer
Integer data contains integers. An array of mesh node ids is an example of integer data.
Data access function
A data access function is a callback function implemented by the participant solver and registered with the participant library. The participant library will call this function when it needs to access the heavyweight data. For a coupled analysis, all required data access functions must be registered prior to _coupled analysis initialization_. For mapping, all required data access functions must be registered prior to _inputs update_.
Data Access Functions
To provide access to the heavyweight data, the participant must also implement data access functions which return the data access structures and register them with the participant library. These functions are called from within the participant library, i.e. they are callback functions.
The following functions may need to be implemented:
- Function to access input scalar data (input scalar data access)
- Function to access input vector data (input vector data access)
- Function to access output scalar data (output scalar data access)
- Function to access output vector data (output vector data access)
- Function to access input complex scalar data (input complex scalar data access)
- Function to access input complex vector data (input complex vector data access)
- Function to access output complex scalar data (output complex scalar data access)
- Function to access output complex vector data (output complex vector data access)
- Function to access surface mesh (surface mesh access)
- Function to access volume mesh (volume mesh access)
- Function to access point cloud (point cloud access)
There is no need to implement and register functions that will not be used. For example, if there are no scalar output variables, it is not necessary to implement and register the function for output scalar variable access.
For information on how to implement data access functions, see Tutorial: Heat Transfer in Square Channel Air Flow.
C++
Data access function prototypes are defined in the following header file:
SystemCouplingParticipant/FunctionTypes.hpp
C
Data access function prototypes are defined in the following header file:
syscFunctionTypes.h
Fortran
Data access function interfaces are defined in the following header file (located either in FortranFreeForm or FortranFixedForm sub-directory, depending on desired Fortran layout):
syscFunctionTypesF.fi
Python
Data access function prototypes closely follow those in C++.
Supported Heavyweight Data Formats
The compatible formats for heavyweight data are:
- Integer arrays
- 32-bit signed integer arrays (for mesh connectivity information)
- C/C++ int32_t
- FORTRAN integer(kind=4)
- Python numpy arrays with datatype
int32
- 64-bit signed integer arrays (for mesh connectivity information)
- C/C++ int64_t
- FORTRAN integer(kind=8)
- Python numpy arrays with datatype
int64
- 16-bit unsigned integer arrays (for mesh connectivity information)
- C/C++ uint16_t
- Python numpy arrays with datatype
uint16
- 64-bit unsigned integer arrays (for mesh connectivity information)
- C/C++ uint64_t
- Python numpy arrays with datatype
uint64
- 32-bit signed integer arrays (for mesh connectivity information)
- Scalar arrays
- 64-bit floating point arrays (for scalar variable field values)
- C/C++ double
- FORTRAN real(kind=8)
- Python numpy arrays with datatype
float64
- 32-bit floating point arrays (for scalar variable field values)
- C/C++ float
- FORTRAN real(kind=4)
- Python numpy arrays with datatype
float32
- 64-bit floating point arrays (for scalar variable field values)
- Vector arrays
- 32-bit floating point arrays (for vector variable field values and for mesh node coordinates)
- C/C++ float
- FORTRAN real(kind=4)
- Python numpy arrays with datatype
float32
- 64-bit floating point arrays (for vector variable field values and for mesh node coordinates)
- C/C++ double
- FORTRAN real(kind=8)
- Python numpy arrays with datatype
float64
- 32-bit floating point arrays (for vector variable field values and for mesh node coordinates)
Complex scalar arrays
- 64-bit floating point complex number arrays (for complex scalar variable field values)
- C++ std::complex<double>
- C/C++ double
- Python numpy arrays with datatype
complex128
- 32-bit floating point complex number arrays (for complex scalar variable field values)
- C++ std::complex<float>
- C/C++ float
- Python numpy arrays with datatype
complex64
Different layouts of vector data are supported
Compact layout
(e.g., a single array storing each vector one after another)
[x0, y0, z0, x1, y1, z1, ..., xN, yN, zN]
Split layout
(e.g., a separate array for each vector component)
[x0, x1, .., xN]
[y0, y1, .., yN]
[z0, z1, .., zN]
Different layouts of complex data are supported
Compact complex scalar layout
(e.g., a single array storing each real and imaginary component one after another)
[real_0, imag_0, real_1, imag_1, ..., real_N, imag_N]
Split complex scalar layout
(e.g., two separate arrays, one for real and one for imaginary components)
[real_0, real_1, ..., real_N]
[imag_0, imag_1, ..., imag_N]
Compact complex compact vector layout
(e.g., a single array storing each real and imaginary component one after another) Note that this is the only layout currently supported in Fortran.
[x0_real, x0_imag, y0_real, y0_imag, z0_real, z0_imag, ..., zN_real, zN_imag]
Split complex compact vector layout
(e.g., a separate array for real and imaginary components)
[x0_real, y0_real, z0_real, x1_real, y1_real, z1_real, ..., xN_real, yN_real, zN_real]
[x0_imag, y0_imag, z0_imag, x1_imag, y1_imag, z1_imag, ..., xN_imag, yN_imag, zN_imag]
Compact complex split vector layout
(e.g., a separate array for each vector component)
[x0_real, x0_imag, x1_real, x1_imag, x2_real, x2_imag, ..., xN_real, xN_imag]
[y0_real, y0_imag, y1_real, y1_imag, y2_real, y2_imag, ..., yN_real, yN_imag]
[z0_real, z0_imag, z1_real, z1_imag, z2_real, z2_imag, ..., zN_real, zN_imag]
Compact complex split vector layout
(e.g., separate arrays vector components and complex components)
[x0_real, x1_real, x2_real, ..., xN_real]
[x0_imag, x1_imag, x2_imag, ..., xN_imag]
[y0_real, y1_real, y2_real, ..., yN_real]
[y0_imag, y1_imag, y2_imag, ..., yN_imag]
[z0_real, z1_real, z2_real, ..., zN_real]
[z0_imag, z1_imag, z2_imag, ..., zN_imag]
Note:
- If vector data is stored in a single array where the vector components are separated (e.g.,
[x0, x1, ..., y0, y1, ..., z0, z1, zN]
), then it can be handled as a special case of split layout. - If vector data is stored in a two-dimensional array, then it can be handled as a special case of one of the above layouts.
- 64-bit floating point complex number arrays (for complex scalar variable field values)
If the participant solver stores heavyweight data in a format compatible with the participant library, then this reduces the need to make unnecessary copies of the data and thus improves the memory efficiency. The participant solver should implement and register data access functions that provide direct access to that heavyweight data.
If the data is stored in a format that is not directly compatible with the participant library, then the participant solver should create compatible heavyweight data structures and register data access functions that provide access to that heavyweight data. In such a case, the participant solver should make sure that this heavyweight data and the original data are synchronized at certains steps.
For mapping, the heavyweight data and the original data should be synchronized at certain steps during mapping:
- Before inputs update, the heavyweight input and output data are expected to be available. Output data, including the mesh data, should contain valid up-to-date values.
- After inputs update completes, all input variable values will be up-to-date in the heavyweight data. The participant should make sure that the latest input values are available in the original data structures before processing that data.
For a coupled analysis, the heavyweight data and the original data should be synchronized at certain steps during the coupled analysis:
- Before coupled analysis initialization is entered, the heavyweight input and output data are expected to be available. Output data, including the mesh data, should contain valid initial values.
- After inputs update completes, all input variable values will be up-to-date in the heavyweight data. The participant should make sure that the latest input values are available in the original data structures before doing solver iterations.
- Before outputs update is entered, the participant should make sure that the heavyweight data contains the updated values for all the output variables, as well as for the mesh.
Array Data Access Structures
Access to heavyweight data is provided via creating data access data structures and implementing data access functions (callback functions) that provide these structures to the participant library. This section discusses the following data access structures:
OutputIntegerData
OutputScalarData
OutputVectorData
InputScalarData
InputVectorData
OutputComplexScalarData
OutputComplexVectorData
InputComplexScalarData
InputComplexVectorData
These structures provide access to data stored in arrays. This section focuses on these structures. These structures can be combined into higher-level structures to provide access to mesh and point cloud data. See Mesh And Point Cloud Data Access for more details.
The array data access structures require three pieces of information during their initialization (depending on the target language, some of this information is deduced automatically and does not need to be explicitly specified):
- Primitive type of data (e.g., float, double, unsigned 16-bit integer, etc.)
- Access to the heavyweight data (e.g., array pointers).
- Extents of the data (e.g., array sizes)
The following examples show how to create data access structures to provide access to heavyweight data to the participant library.
Integer Data Structure Examples
Example: Output Integer Data
In this example, the mesh node ids are stored as an array of unsigned 64-bit integer values. To provide access to this variable, create the OutputIntegerData
structure as shown below.
C++
Note that for C++, if data is stored in STL Vector container, that container can be used directly. See Example Input Scalar in an STL Vector Container (C++ only).
C
For C, a helper function syscGetOutputIntegerDataUInt64
is provided for this case.
Fortran
For Fortran, a helper function syscGetOutputIntegerDataF
is provided for this case.
Python
For Python, any C-style array can be used to create OutputIntegerData
object, with the help of ctypes module.
Note that for Python, if data is stored in a Numpy array container, that container can be used directly. See Example Output Integer Data in a Numpy Array Container (Python only).
Scalar Data Structure Examples
Example: Input Scalar Data
In this example, the participant solver has values of an input scalar variable stored in memory as a double-precision array. To provide access to this variable, create the InputScalarData
structure as shown below.
C++
C
For C, a helper function syscGetInputScalarDataDouble
is provided for this case.
Fortran
For Fortran, a helper function syscGetInputScalarDataF
is provided for this case.
Python
For Python, numpy arrays can be passed in to create InputScalarData
object.
Example: Output Scalar Data
In this example, the participant solver has values of an output scalar variable stored in memory as a single-precision array. To provide access to this variable, create the OutputScalarData
structure as shown below.
C++
C
For C, a helper function syscGetOutputScalarDataFloat
is provided for this case.
Fortran
For Fortran, a helper function syscGetOutputScalarDataF
is provided for this case.
Python
For Python, numpy arrays can be passed in to create OutputScalarData
object.
Vector Data Structure Examples
Vector data storage can be categorized as either compact or split storage. Examples are provided for both categories.
Example: Compact Input Vector Data In a One-Dimensional Array
In this example, the participant solver has values of an input vector variable stored in memory as a compact one-dimensional array. To provide access to this variable, create the InputVectorData
structure as shown below.
Note that the size provided is 10 — there are 10 vectors, each with 3 components. So, the total size of the array in memory is 30, but the size passed in here must be 10, not 30.
C++
C
For C, a helper function syscGetInputVectorDataCompactDouble
is provided for this case.
Fortran
For Fortran, a helper function syscGetInputVectorDataCompactF
is provided for this case.
Python
For Python, numpy arrays can be passed in to create InputVectorData
object.
Example: Compact Output Vector Data In a Two-Dimensional Array
In this example, the participant solver has values of an output vector variable stored in memory as a compact two-dimensional array. To provide access to this variable, create the OutputVectorData
structure as shown below.
Note that the size provided is 2 — there are 2 vectors, each with 3 components.
C++
C
For C, a helper function syscGetOutputVectorDataCompactFloat
is provided for this case.
Fortran
For Fortran, a helper function syscGetOutputVectorDataCompactF
is provided for this case.
Note that Fortran stores data in column-major order, so compactly stored two-dimensional array has component as first index and vector as second index.
Python
For Python, numpy arrays can be passed in to create OutputVectorData
object.
Example: Split Input Vector Data In Separate Arrays
In this example, the participant solver has values of an input vector variable stored in memory as three separate arrays, one array per component. To provide access to this variable, create the InputVectorData
structure as shown below.
Note that the size provided is 4 — there are 4 vectors, each with 3 components.
C++
C
For C, a helper function syscGetInputVectorDataSplitFloat
is provided for this case.
Fortran
For Fortran, a helper function syscGetInputVectorDataSplitF
is provided for this case.
Python
For Python, numpy arrays can be passed in to create InputVectorData
object.
Example: Split Output Vector Data In a Two-Dimensional Array
In this example, the participant solver has values of an output vector variable stored in memory as a split two-dimensional array. To provide access to this variable, create the OutputVectorData
structure as shown below.
Note that the size provided is 2 — there are 2 vectors, each with 3 components.
C++
C
For C, a helper function syscGetOutputVectorDataSplitDouble
is provided for this case.
Fortran
For Fortran, a helper function syscGetOutputVectorDataSplitF
is provided for this case.
Note that Fortran stores data in column-major order, so non-compactly stored two-dimensional array has the vector as the first index and the vector component as the second index.
Python
For Python, numpy arrays can be passed in to create OutputVectorData
object.
Example: Compact Output Complex Vector Data In a Two-Dimensional Array, Compact Layout
In this example, the participant solver has values of an output vector variable stored in memory as a split two-dimensional array. To provide access to this variable, create the OutputComplexVectorData
structure as shown below.
Note that the size provided is 2 — there are 2 vectors, each with 3 components, and each component is a complex number.
C++
C
For C, a helper function syscGetOutputCompactComplexCompactVectorDataDouble
is provided for this case.
Fortran
For Fortran, a helper function syscGetOutputComplexVectorDataCompactF
is provided for this case.
Note that Fortran stores data in column-major order, so compactly stored two-dimensional array has the vector component as the first index and the vector as the second index.
Python
For Python, numpy arrays can be passed in to create OutputComplexVectorData
object.
Other Examples
Example Input Scalar in an STL Vector Container (C++ only)
In this example, the participant solver has an input scalar variable stored in memory as an STL vector.
To provide access to this array, create the InputScalarData
structure as shown below.
Note that the size does not need to be provided because it can be automatically deduced from the STL vector size.
C++
Example Output Integer Data in a Numpy Array Container (Python only)
In this example, the participant solver has an output integer data stored in memory as a numpy array.
To provide access to this array, create the OutputIntegerData
structure as shown below.
Note that the size does not need to be provided because it can be automatically deduced from the Numpy array size.