Public Catalogs‎ > ‎

README

*******************************************************************************

INTRODUCTION

This is a public void catalog of the following populations:

 + SDSS DR7 in redshift space and comoving space ("sdss_dr7")
 + SDSS DR9 in redshift space and comoving space ("sdss_dr9")
 + SDSS DR10 in redshift space and comoving space ("sdss_dr10")
 + 1024^3 particle, 1 Gpc/h Dark Matter Simulation ("mock1024")
 + Dark Matter Halos in the above simulation
 + HOD Mocks drawn from the above simulation
 + HOD Mocks drawn from a 4096^3 particle, 4 Gpc/h simulation ("lanl")
 + HOD Mocks from 98 2LPT runs matching SDSS DR9 (Manera et al. 2012)*
 + masked SDSS (un)constrained realizations ("masked3D")

For information regarding the construction of this catalog, please see Sutter 
et al. "A public void catalog from the SDSS DR7 galaxy redshift surveys based 
on the watershed transform" (arXiv: 1207.2524), Sutter et al. "VIDE: 
The Void IDentification and Examination toolkit" (arXiv:1406.1191),
and the News & Updates section of http://www.cosmicvoids.net for corrections 
and clarifications.

If you use the DR7 voids, please cite arXiv: 1207.2524
If you use the DR9 voids, please cite arXiv: 1310.7155
If you use the DR10 voids, please cite arXiv: 1404.5618
If you use the "mock1024" voids, please cite arXiv: 1309.5087
If you use the "lanl" voids, please cite arXiv: 1309.5087
If you use the "2lpt" voids, please cite arXiv: 1310.7155
If you use the "masked3D" voids, please contact florent.leclercq@polytechnique.org for more information

For questions, please contact Paul Sutter (sutter@iap.fr). Note that 
requests for *additional* processing and/or analysis will be treated 
as invitations for collaboration and joint authorship!

* Due to space constraints on our current server this is only available 
  by request. We are trying to resolve the situation. Sorry!

*******************************************************************************

VERSION HISTORY

2015.03.31: Added masked3D

2014.06.18 : Fixed bug in report of mean separation in sample_info.txt
             Added sdss_dr10
             Put void centers (x,y,z) for observations in correct coordinates

2013.10.25 : New layout: sdss_dr7 and sdss_dr9 now contain two subdirectories, 
               "redshift" and "comoving", corresponding to the coordinate 
               system in which we identified the respective voids.

             Removal of alternate catalogs that trim on hierarchy level;
               we now just provide two catalogs: a "default" catalog with 
               only parent voids with central density < 0.2 mean, and a 
               fully untrimmed version.

             Guide to trimming the catalog added to README.

             Number of particles, number of boundary particles, survey volume,
                estimated mean particle separation added to sample_info.txt

             Inclusion of 2LPT mocks and masked N-body mocks

2013.09.19 : "_nocut_" files now replaced by "untrimmed_" files and contain
             full catalog information (barycenters, sky positions, shapes, etc.)

             Voids that contain subvoids are now removed by default 
               (n.b. this really only affects high-resolution mocks)
             All voids, including "parent" voids, are in the "untrimmed_" files
             centers_ file outputs parent ID and number of sub-voids

             Improved shape estimation

             Addition of mock catalogs
 
             Addition of DR9 voids

             Inclusion of particle volume file

             Inclusion of source catalogs

2013.02.20 : More accurate calculation of mean galaxy separation and hence 
               minimum acceptable void radius, but please USE CAUTION when
               using the smallest voids!
             Fix so that central density cut is actually applied
             Correct units in centers_ file header
             Inclusion of PRELIMINARY ellipticity information

2012.11.17 : Removal of NetCDF file
             Inclusion of member galaxy RA, Dec, redshift, and catalog ID
               in the zobov binary particle file.
             New layout - no more "all" and "central" directories
             List of void centers now includes density contrast
             Inclusion of dumpVoidParticles.c in release package
             Improvements to dumpVoidParticle to report RA, Dec, etc.
             sample_info.txt file now contains all sample information
             Corrected "central" sample voids
             Added additional small voids to "all" sample
             More consistent handling of redshift boundaries

2012.08.16 : new layout - "lcdm" now part of sdss/dr7 subdirectory
             added void ID column to centers.txt and sky_positions.txt
             added NetCDF file with void galaxy RA, Dec, and redshift
             added sample_info.txt files which provide basic sample info
             improved dumpVoidParticles.c
             
2012.07.04 : Initial release.


*******************************************************************************

LAYOUT OF THIS CATALOG

The void catalog for each sample is given its own top-level directory in the catalog. The directories associated with each catalog are:

 + sdss_dr7  : SDSS DR7 Main Sample and LRGs
               There are two subdirectories: redshift and comoving, 
                 corresponding to the coordinate system in which we 
                 identified the voids.
               We divide this catalog into subdirectories based on the 
               volume-limited subsamples of the source catalog, labeled as 
               sample_lss.dr72[sample_name].dat. 

 + sdss_dr9 : SDSS DR9 CMASS
               There is a single subdirectory: comoving.
               We divide this catalog into subdirectories based on the 
               volume-limited subsamples of the source catalog, labeled as 
               sample_lss.dr9cmass[sample_name].dat. 

 + mock1024 : Dark Matter Simulation of 1024^3 particles in 1 Gpc/h box
              All particles are kept in real space at redshift 0 
              and peculiar velocities are not included.

              We divide this catalog into subdirectories based on different
              ways of tracing the dark matter population:

              mt_ssNNNNN : Random subsampling of the dark matter particles.
                           NNNNN is the mean number density in h^3 Mpc^-3.

              mt_halos_minNNNNN : Just taking halo centers, only including 
                                  halos with minimum mass NNNNN.
                                  "none" means all halos.

              mt_hod_XXXXX : Mock galaxy population generated with Jeremy 
                             Tinker's HOD code. Tuned to the observational
                             HOD fits to galaxy population XXXXX.

              The "_d00" suffix is used to indicate any subvolumes, which 
              we are not using for this catalog.

 + lanl : Dark Matter Simulation of 4096^3 particles in 4 Gpc/h box
          This was used to generate a single HOD mock at z=0.5.
          Peculiar velocities are included.
          Note that despite the names below, these are HOD mocks! They were 
            processed slightly differently and ended up with a different name.
 
          We divide this catalog into subdirectories as follows:

          lanl_halos_minnone_pv : HOD Mock of dr9mid in full cubic volume

          masked/masked_lanl_halos_minnone_pv_z0.5_rNN : Masked HOD mock 
            with 6 different rotations of mask within volume. Galaxies 
            placed on lightcone measured from center of box.
 
 + 2lpt : HOD 2LPT Mocks with the same survey geometry as SDSS DR9.
          See Manera et al. (2012) for details.
 
          We divide this catalog into subdirectories numbered 3001-3099.
          
 + masked3D : Centers and sky positions ONLY for constrained and 
                unconstrained simulations of arXiv:1410.0355, due to the 
                difficulty in constructing the source simulations.
              WARNING: NOT COMPATIBLE WITH VIDE TOOLKIT!
              Contact florent.leclercq@polytechnique.org for more information.

In addition, there are the following top-level directories: 

 + source_catalogs: ASCII files of data populations used for void finding 
                    (at this time we cannot include simulation populations
                     due to size contraints).

 + figures: contains data files necessary to reproduce the figures found in the 2012 paper

 + tools: routines and useful tools for reading the catalog. The program dumpVoidParticles has an example input file and can be used to extract void galaxies.

-------------------------------------------------------------------------------

The following files describe the sample, ZOBOV catalog, and void zone and galaxy information:

 + sample_info.txt: Small text file which lists some basic info of the sample
 
 + zobov_slice_lss.dr72[sample_name].dat: Binary file with the galaxy positions, RA, Dec, redshift, and unique catalog ID. Note that this file contains *all* particles in the particular sample, not just void galaxies. See the dumpVoidParticles.c routine for parsing this file and extracting void galaxies.
  
 + voidDesc_[all,central]_[sample_name].out: ASCII file with void indices and 
     basic void properties in ZOBOV-normalized units.

 + voidZone_[sample_name].out: Binary file which links voids to zones.

 + voidPart_[sample_name].out: Binary file which links zones to particles.

 + vol_[sample_name].out: Binary file with particle local volumes

To parse these files, see the dumpVoidParticles.c routine in the tools/ directory, or visit the ZOBOV help page at
http://skysrv.pha.jhu.edu/~neyrinck/voboz/zobovhelp.html

-------------------------------------------------------------------------------

In addition, we provide the following derived void information:

 + centers_[all, central]_[sample_name].txt: ASCII file with void centers, 
     volume, effective radius, void ID, density contrast, etc.

 + sky_positions_[all, central]_[sample_name].txt: ASCII file with void RA, 
     dec, effective radius, and void ID

 + shapes_[all, central]_[sample_name].txt: ASCII file with best-fit 
     eigenvalues and eigenvectors for each void, in same coordinate system 
     as x,y,z values in centers_ file

-------------------------------------------------------------------------------

In addition to the [all,central] version of each file above, there are
two versions of each catalog:

 + no prefix: only parent voids, density cut applied (default catalog)
 
 + untrimmed: all voids, density cut not applied

------------------------------------------------------------------------------

The figures/ directory contains data files which can be used to reproduce 
various figures in the paper. The file names indicate the figure and whether
they are from the "central" or "all" catalog:

  fig5_[central,all].dat: Size distributions. 
    ASCII file with 7 columns: radius bin, and one column of number counts 
    for each sample 

  fig6_[central,all].dat: Redshift distributions.
    ASCII file with 7 columns: redshift bin, and one column of number counts 
    for each sample

  fig7_[10-15,20-25,50-55,90-95]_[central,all].dat: Radial profiles. 
    One file for each stack (the number indicates the radius range of that 
    stack). Variable number of columns depending on which samples have voids 
    in that stack. The first column is the radius in normalized units, and 
    subsequent columns are normalized number densities. The header indicates 
    which samples are in this file.

  fig8_[10-15,20-25,50-55,90-95]_central.npy: Projections. 
    One file for each stack (the number indicates the radius range of that 
    stack). We supply projections for only the central catalog. NumPy array
    of the output of matplotlib's hexbin function, which was used to generate
    the plots. 


*******************************************************************************

FILTERING THE CATALOG

We have developed a philosophy about this void catalog in which we attempt to report as many voids as possible and let you, the user, apply cuts and filters as you see appropriate. We have a set of filters that we apply for our own analysis, and for historical reasons this has become the "default" catalog. We also provide the full unfiltered catalog (except for a hard minimum size cut, to be discussed below) so you can create your own version of the catalog. 

Some cuts you may consider applying:

- Size: We only provide voids with effective radii greater than the mean particle separation. However, the smallest voids are likely highly unreliable. Thus, you may want to only take voids with two times the minimum radius. You can find the radius in the "centers_" files.

- Minimum density: The watershed algorithm reports all basins, even if they have densities higher than you may like. You can, for example, only take voids with minimum Voronoi cell density < 0.2 or 0.3 times the mean density (as in Nadathur et al. 2013). Since minimum density scales inversely with void size, this also effectively removes most small voids. However, this quantity is very noisy, so your cuts may be more/less aggressive than you think. The minimum density can be found in the "voidDesc_" files. Since this is a raw ZOBOV output, it is expressed in units normalized to the total number of particles, including boundary particles. So you can either a) correct for the presence of boundary particles by rescaling by a factor of (n_g/n_tot)^3, or b) just look at the distribution of minimum densities and make a cut as you see fit. You can find n_g and t_tot in the sample_info.txt file.

- Central density: This is similar in spirit to the minimum density above, but is typically computed in spheres about the barycenter. For the default catalog, we take spheres of 1/4 the void effective radius. This tends to affect larger voids, and becomes less effective for smaller voids (since we start to probe below the mean particle separation). If you want to calculate this on your own, then you have to use the dumpVoidParticles tool to extract the member particles.

- Density contrast: This is the ratio between the density on the void ridgeline and the minimum density. You can compare this to, say, the probability of getting that density ratio in a Poisson distribution (Neyrinck 2008), but this procedure is not necessarily appropriate for sparse, biased populations. You can find this quantity in both the voidDesc_ and centers_ files.

- Tree level: There is a full hierarchy of voids, subvoids, sub-subvoids, etc.. You may wish to take only top-level "parent" voids, or only leaf-level "children" voids. The default catalog contains only parent voids. You can find the tree level in the centers_ files.

Our default catalog contains only top-level parent voids with central densities < 0.2 times the mean density. If you are happy with these cuts, then you can just use the un-prefixed catalog.

If you want to make your own cuts, then start with the "untrimmed_" catalogs and help yourself. 

In addition, we also have the "all" and "central" catalog division. 

We would like to thank Seshadri Nadathur; our conversations with him prompted the creation of this post.


*******************************************************************************

SPECIAL NOTES: 


*******************************************************************************

Comments