PlanarBoundary Class Reference

#include <ERF_PlanarBoundary.H>

Collaboration diagram for PlanarBoundary:

Classes
struct	Buffer

Public Member Functions
void	define (const amrex::BoxArray &ba3d, const amrex::BoxArray &ba2d, const amrex::DistributionMapping &dm, int surface_index, bool is_low=true, int normal_dir=2)
void	fill (amrex::MultiFab &mf, const amrex::Periodicity &period)
void	gather_surface (const amrex::MultiFab &mf, amrex::MultiFab &dst, int scomp, int dcomp, int ncomp) const
const amrex::BoxArray &	surface_boxes () const
	Cell-centered surface copies of the planar boxes, without duplicates. More...
const amrex::DistributionMapping &	surface_dm () const
	Ranks owning the surface copies: each is on the rank of the planar box it is a copy of. More...
const amrex::Vector< int > &	surface_index () const
	Index of each surface copy in the planar BoxArray. More...
AMREX_GPU_HOST bool	is_surface_copy (const int planar_index) const
	Whether a planar-box index is the computed copy that touches the surface. More...

Private Member Functions
amrex::MultiFab &	buffer (const amrex::MultiFab &mf)

Private Attributes
int	m_nplanar {0}
amrex::BoxArray	m_ba_sfc
amrex::DistributionMapping	m_dm_sfc
amrex::Vector< int >	m_src_index
amrex::Vector< Buffer >	m_buffers

Detailed Description

Ghost-cell fill for planar (2D) MultiFabs built on the collapse of a 3D BoxArray along a surface-normal direction.

The surface-layer fields and the MOST averages are defined on one 2D box per 3D box. When the 3D BoxArray is split in the normal direction, several 3D boxes collapse onto the same 2D box, so the planar BoxArray holds duplicate boxes with overlapping valid regions, and only the copy that belongs to the 3D box touching the surface is ever computed. A FillBoundary on such a MultiFab may fill a ghost cell from an uncomputed copy.

define() records which planar boxes are those surface copies. fill() copies them into a buffer without duplicates and ParallelCopies the buffer onto every planar box, valid region and ghost cells alike. Unlike FillBoundary it therefore also overwrites the valid region of the uncomputed copies, so that any copy can be read afterwards. Without duplicates fill() is a plain FillBoundary.

The buffer is kept for each target layout, index type, and number of components, so only the first fill of each kind allocates. For a face-centered MultiFab (the MOST velocity averages) neighbouring surface boxes share a face in the buffer. Both should hold the same value there, but nothing enforces it, so fill() gives the shared faces a single value before it copies: the lowest global box index wins, which does not depend on the decomposition or the rank count, and every copy of the field then agrees.

A level none of whose boxes reaches the surface (a fine level away from the surface) has no computed copy to fill from, and fill() leaves the MultiFab as it is.

Member Function Documentation

buffer()

MultiFab & PlanarBoundary::buffer ( const amrex::MultiFab &  mf )

private

Gather buffer for one target layout, index type, and number of components, allocated on first use. The source MultiFab list avoids rebuilding the layout key on every fill.

Parameters

[in]

mf

planar MultiFab to buffer

{
    const IndexType ixtype = mf.ixType();
    const int ncomp = mf.nComp();
 
    // define() clears m_buffers whenever the underlying layout changes, so a
    // source pointer is a stable per-field cache key for the lifetime of this
    // PlanarBoundary definition.
    for (auto& b : m_buffers) {
        if (b.ixtype == ixtype && b.ncomp == ncomp &&
            std::find(b.sources.begin(), b.sources.end(), &mf) != b.sources.end()) {
            return *b.mf;
        }
    }
 
    // Only an unfamiliar source field needs the more expensive derived-layout
    // construction below.  Fields with the same layout continue to share one
    // gather buffer.
    BoxList bl_sfc(ixtype);
    Vector<int> pmap;
    for (int src : m_src_index) {
        bl_sfc.push_back(mf.boxArray()[src]);
        pmap.push_back(mf.DistributionMap()[src]);
    }
    BoxArray ba(std::move(bl_sfc));
    DistributionMapping dm(std::move(pmap));
 
    for (auto& b : m_buffers) {
        if (b.ixtype == ixtype && b.ncomp == ncomp && b.ba == ba && b.dm == dm) {
            b.sources.push_back(&mf);
            return *b.mf;
        }
    }
    m_buffers.push_back(Buffer{ixtype, ncomp, ba, dm, {&mf},
                               std::make_unique<MultiFab>(ba, dm, ncomp, 0)});
    return *m_buffers.back().mf;
}

define()

void PlanarBoundary::define	(	const amrex::BoxArray &	ba3d,
		const amrex::BoxArray &	ba2d,
		const amrex::DistributionMapping &	dm,
		int	surface_index,
		bool	is_low = true,
		int	normal_dir = 2
	)

Record the surface copies of a planar BoxArray.

Parameters

[in]	ba3d	3D BoxArray the planar BoxArray was collapsed from
[in]	ba2d	planar BoxArray, one box per box of ba3d and in the same order
[in]	dm	DistributionMapping shared by ba3d and ba2d
[in]	surface_index	index of the surface cell in the normal direction
[in]	is_low	whether the surface is the low side of the domain
[in]	normal_dir	normal direction of the surface

Record the surface copies of a planar BoxArray (see ERF_PlanarBoundary.H).

Parameters

[in]	ba3d	3D BoxArray the planar BoxArray was collapsed from
[in]	ba2d	planar BoxArray, one box per box of ba3d and in the same order
[in]	dm	DistributionMapping shared by ba3d and ba2d
[in]	surface_index	index of the surface cell in the normal direction
[in]	is_low	whether the surface is the low side of the domain
[in]	normal_dir	normal direction of the surface

{
    AMREX_ALWAYS_ASSERT_WITH_MESSAGE(ba2d.size() == ba3d.size(),
        "PlanarBoundary::define: the planar BoxArray must hold one box per 3D box");
 
    m_nplanar = static_cast<int>(ba2d.size());
    m_src_index.clear();
    m_buffers.clear();
 
    // The planar boxes are taken as they are, whatever index they were collapsed to
    BoxList bl_sfc(IndexType::TheCellType());
    Vector<int> pmap;
    for (int ib = 0; ib < m_nplanar; ++ib) {
        const bool touches_surface = is_low
            ? (ba3d[ib].smallEnd(normal_dir) == surface_index)
            : (ba3d[ib].bigEnd(normal_dir) == surface_index);
        if (touches_surface) {
            bl_sfc.push_back(enclosedCells(ba2d[ib]));
            pmap.push_back(dm[ib]);
            m_src_index.push_back(ib);
        }
    }
    m_ba_sfc = BoxArray(std::move(bl_sfc));
    m_dm_sfc = DistributionMapping(std::move(pmap));
 
    // Exactly one computed copy per column: the surface boxes must not overlap
    AMREX_ALWAYS_ASSERT_WITH_MESSAGE(m_ba_sfc.isDisjoint(),
        "PlanarBoundary::define: the boxes touching the surface overlap in the plane, so a "
        "planar field would have more than one computed copy per column");
}

Here is the call graph for this function:

fill()

void PlanarBoundary::fill	(	amrex::MultiFab &	mf,
		const amrex::Periodicity &	period
	)

Fill every copy of a planar MultiFab, valid region and ghost cells, from the surface copies.

Parameters

[in,out]	mf	planar MultiFab on ba2d, of any index type and number of components
[in]	period	periodicity of the level

Fill every copy of a planar MultiFab, valid region and ghost cells, from the surface copies (see ERF_PlanarBoundary.H).

Parameters

[in,out]	mf	planar MultiFab on the planar BoxArray given to define
[in]	period	periodicity of the level

{
    AMREX_ALWAYS_ASSERT_WITH_MESSAGE(static_cast<int>(mf.size()) == m_nplanar,
        "PlanarBoundary::fill: the MultiFab is not on the planar BoxArray given to define");
 
    const int nsfc = static_cast<int>(m_src_index.size());
 
    // Every planar box is a surface box: no duplicates, FillBoundary is well defined
    if (nsfc == m_nplanar) {
        mf.FillBoundary(period);
        return;
    }
 
    // No box on this level reaches the surface, so no copy holds computed data to fill
    // from; leave the MultiFab as it is
    if (nsfc == 0) { return; }
 
    const int ncomp = mf.nComp();
    MultiFab& buf = buffer(mf);
    gather_surface(mf, buf, 0, 0, ncomp);
 
    // A face-centered buffer's boxes share a face with their neighbours, and the gather above
    // takes each box's face from its own surface copy, so two boxes can hold different values
    // there.  Give every shared face one value before the ParallelCopy -- OverrideSync's
    // precedence is the lowest global box index, so the result does not depend on the
    // decomposition or the rank count -- and every copy of mf then ends up with the same
    // value.  Returns immediately for cell-centered data, which has no shared faces.
    buf.OverrideSync(period);
 
    // Fill every copy, valid region and ghost cells, from the computed surface copies
    mf.ParallelCopy(buf, 0, 0, ncomp, IntVect(0), mf.nGrowVect(), period);
}

Here is the call graph for this function:

gather_surface()

void PlanarBoundary::gather_surface	(	const amrex::MultiFab &	mf,
		amrex::MultiFab &	dst,
		int	scomp,
		int	dcomp,
		int	ncomp
	)		const

Copy the computed surface copies of a planar MultiFab into a MultiFab without duplicates.

define() gives each surface copy the rank of the planar box it is a copy of, so every box of dst is filled from a FAB of mf that lives on the same rank and the gather needs no communication. Reading the surface copies directly, rather than any copy of the field, means the caller need not have filled the duplicates with fill().

Parameters

[in]	mf	planar MultiFab on the planar BoxArray given to define
[out]	dst	MultiFab on surface_boxes() converted to the index type of mf, and on surface_dm()
[in]	scomp	first component to read from mf
[in]	dcomp	first component to write in dst
[in]	ncomp	number of components

Copy the computed surface copies of a planar MultiFab into a MultiFab without duplicates (see ERF_PlanarBoundary.H).

Parameters

[in]	mf	planar MultiFab on the planar BoxArray given to define
[out]	dst	MultiFab on the surface boxes, in the index type of mf
[in]	scomp	first component to read from mf
[in]	dcomp	first component to write in dst
[in]	ncomp	number of components

{
    AMREX_ALWAYS_ASSERT_WITH_MESSAGE(static_cast<int>(mf.size()) == m_nplanar,
        "PlanarBoundary::gather_surface: the MultiFab is not on the planar BoxArray given to define");
    AMREX_ALWAYS_ASSERT_WITH_MESSAGE(dst.size() == static_cast<Long>(m_src_index.size()),
        "PlanarBoundary::gather_surface: the destination is not on the surface boxes");
 
    for (MFIter mfi(dst); mfi.isValid(); ++mfi) {
        const Box& bx = mfi.validbox();
        const int src = m_src_index[mfi.index()];
        // The surface copy must be the planar box with the same footprint, on this rank
        AMREX_ALWAYS_ASSERT_WITH_MESSAGE(bx == mf.boxArray()[src] &&
                                         mf.DistributionMap()[src] == ParallelDescriptor::MyProc(),
            "PlanarBoundary::gather_surface: surface box does not match its planar box");
        dst[mfi].copy<RunOn::Device>(mf[src], bx, scomp, bx, dcomp, ncomp);
    }
}

Referenced by SurfaceLayer::surface_sum().

Here is the call graph for this function:

Here is the caller graph for this function:

is_surface_copy()

AMREX_GPU_HOST bool PlanarBoundary::is_surface_copy ( const int  planar_index ) const

inline

Whether a planar-box index is the computed copy that touches the surface.

    {
        return std::binary_search(m_src_index.begin(), m_src_index.end(), planar_index);
    }

surface_boxes()

const amrex::BoxArray& PlanarBoundary::surface_boxes ( ) const

inline

Cell-centered surface copies of the planar boxes, without duplicates.

{ return m_ba_sfc; }

Referenced by SurfaceLayer::surface_sum().

Here is the caller graph for this function:

surface_dm()

const amrex::DistributionMapping& PlanarBoundary::surface_dm ( ) const

inline

Ranks owning the surface copies: each is on the rank of the planar box it is a copy of.

{ return m_dm_sfc; }

Referenced by SurfaceLayer::surface_sum().

Here is the caller graph for this function:

surface_index()

const amrex::Vector<int>& PlanarBoundary::surface_index ( ) const

inline

Index of each surface copy in the planar BoxArray.

{ return m_src_index; }

Member Data Documentation

m_ba_sfc

amrex::BoxArray PlanarBoundary::m_ba_sfc

private

Referenced by surface_boxes().

m_buffers

amrex::Vector<Buffer> PlanarBoundary::m_buffers

private

m_dm_sfc

amrex::DistributionMapping PlanarBoundary::m_dm_sfc

private

Referenced by surface_dm().

m_nplanar

int PlanarBoundary::m_nplanar {0}

private

m_src_index

amrex::Vector<int> PlanarBoundary::m_src_index

private

Referenced by is_surface_copy(), and surface_index().

---

The documentation for this class was generated from the following files:

• Source/Utils/ERF_PlanarBoundary.H
• Source/Utils/ERF_PlanarBoundary.cpp