Full Changelog

3.6dev (XX/XX/XXXX)

3.5 (01/02/2021)

  • Update FSF module (new Moffat model, correct fsf centering in get_2darray)

  • Correct bug in mpdaf.obj.Spectrum.mean()

  • Solve rounding problem in cubemosaic

  • Add wcs.get_center() and wcs.coord()

  • Allow reading not-projected pixtable (when astrometry was not applied)

  • Fix documentation link

  • Correct a bug in mpdaf.obj.image.peak()

  • Change NeV1238 and NeV1248 to NV1238, NV1248 in mpdaf.sdetect.linelist

  • Add a parameter in mpdaf.sdetect.linelist.get_emlines to exclude a wavelength range (useful for AO spectra with masked Na region)

3.4 (17/01/2020)

  • Update fsf (combine_fsf, fwhm_moffat2gauss, documentation)

  • Mask NaN in muselet

  • Python 3.6+ is required.

3.3 (09/11/2019)

  • In mpdaf.obj.cube.get_image add an option to compute background from a median filter. Propagate this addition to mpdaf.sdetect.source.add_narrow_band_image

  • In mpdaf.sdetect.get_emlines, encode the DNAME column in bytes to make it easier to export tables with this into FITS.

  • Remove lo and up columns (not used) and add a vdisp column (to identify AGN type lines) in mpdaf.sdetect.linelist.

  • Add OIII1660, replace CII 2326 line by a quadruplet in mpdaf.sdetect.linelist.

  • Fix conversion to pixel coordinates in mpdaf.sdetect.Catalog.select.

  • Avoid crashes with mpdaf.sdetect.Source.add_table when some extension are missing in the Source. By default, select_in and col_dist are not used.

  • Use lowercase extension names when writing a PixTable, otherwise the DRS is not able to read it. This fixes a regression introduced in a previous version.

  • Add mpdaf.drs.PixTable.get_row to access pixtable rows.

3.2 (22/05/2019)

Removal of deprecated code:

  • Spectrum.line_gauss_fit has been removed, it has been deprecated since v2.3.

  • Source.add_masks has been removed, it has been deprecated since v1.2.

  • The self-calibration code has been removed (PixTable.selfcalibrate and PixTable.sky_ref). The self-calibration method is available in the DRS since version 2.4, with the autocalib="deepfield" parameter. This should be preferred as it is more efficient (no need to save the Pixtable to read it in MPDAF), includes a few bug fixes, and allows one to use the DRS sky-subtraction after the autocalib. New DRS versions may also include more features.

3.1 (03/04/2019)

Muselet

Numerous changes to muselet. [!160]

  • Improved treatment of extended sources: expand image sizes, as needed, returns sextractor segmentation mask.

  • Each narrow band now uses a corresponding weight map, not a global one.

  • More robust cleaning of false detections outside FoV.

  • Changes to merging algorithms. Results in slightly more objects (i.e. more fragmentation, not necessarily more detections).

  • Fixed “too many files open error” (hopefully).

  • Merged “cont” and “sing” catalogs into one “object” catalog.

  • Performance also improved via multiprocessing (but disabled by default).

3.0 (10/19/2018)

MPDAF now requires Python 3.5 or later.

Pixtable

  • The self-calibration method is deprecated, as it is available in the DRS since version 2.4, with the autocalib="deepfield" parameter.

  • Add the plot_autocal_factors function to plot the autocalib corrections. [!138]

Muselet

  • Fix Muselet issue with AO cubes. [#513]

Cube, Image, Spectrum

  • Add mpdaf.obj.Cube.get_band_image to create images with the same filters as the DRS (this includes HST, SDSS, or other filters). [!146]

  • Add rgb plotting function plot_rgb and a method to Source objects (mpdaf.sdetect.Source.show_rgb). [!123]

  • Adds parameter to avoid the float32/float64 conversions: by default arrays in Spectrum/Image/Cube are converted to float32 when saving, to save some disk space. This can be avoided with convert_float32=False. And when reading files, a float64 conversion can now be avoided with convert_float64=False. [!126, !!135]

  • Remove duplicate air/vacuum conversion functions, now available as mpdaf.obj.airtovac and mpdaf.obj.vactoair. [!127]

  • Fix subcube when using np.ma.nomask [!128].

  • Check that FITS file’s naxis is compatible with the class [#505, !129]

  • Fix the copy of FITS header to respect the initial order of keywords. [!137]

Source, Catalog

  • Support the use of metadata in Source tables and Catalog. Keywords IDNAME, RANAME and DECNAME can be used to specify the column names for the ID, RA and DEC columns that are used in several Catalog methods. [!124]

  • Speedup get_FSF_from_cube_keywords. [!131]

  • Add cube.min(). [!133]

  • Correct bug in source crackz execution.

  • Correct bug in match to return only one catalog when option full_output is False. [!139]

  • Fix get_emlines to use unicode for line type. [!142]

Misc

  • Fix pickling of MPDAF objects when the header does not match the data. [!141]

  • Fix warnings with recent Numpy versions.

2.5.1 (16/03/2018)

  • Fix Spectrum.gauss_fit to always return a positive fwhm. [#501]

  • Fix several issues with the CubeList C extension:

    • Avoid segfault with -std=c99.

    • Fix compilation when NAN is not defined.

    • Fix the extension when not using OpenMP, which is still the case with Clang. Support for OpenMP with Clang will be addressed in the future.

  • Changes to Cube.get_image:

    • It’s now possible to use any method from the Cube object that reduces the data on the wavelength axis to get an image, e.g. mean, sum, max…

    • The way to compute the background when asking for a summed image has changed. Before, it was computed averaging the pixel values of regions ‘below’ and ‘above’ the kept wavelengths; now it is calculated as the mean of the average in each region. This may lead to slightly different results when working at the edge of the spectrum, when the width of the regions are different.

2.5 (02/03/2018)

  • Numpy is now installed with setup_requires, which means that it is no more required to run the setup, and it can be installed directly as a dependency.

  • Allow one to pickle MPDAF objects (Cube, Image, Spectrum, Source). This makes it much easier to use these MPDAF objects with multiprocessing as they can now be passed transparently to the subprocesses.

  • Allow one to specify the wavelength range in Cube.subcube_circle_aperture.

  • Speedup create_psf_cube, moffat_image and gauss_image.

  • Add option to plot images with world coordinates axes, using astropy.visualization.wcsaxes. This can be used with Image.plot(..., use_wcs=True)

Sources

  • Better completions for Source attributes, and for keys in IPython.

  • Allow one to load a “fieldmap” from a dedicated file (in Source.add_FSF).

  • New method to get FSF keywords from a Source (Source.get_FSF).

Catalog

  • Remove the index that was added by default on the ‘ID’ column, as it was causing errors with some operations due to bugs in the Astropy implementations. Indexes can still be added manually if needed.

  • Allow one to pass additional Ellipse arguments to Catalog.plot_symb.

  • Allow one to export coordinates from their columns to a astropy.coordinates.SkyCoord object (Catalog.to_skycoord) and to ds9 region file (Catalog.to_ds9_regions).

  • New methods Catalog.nearest, to get the nearest sources with respect to a given coordinate, and Catalog.match3Dline, to match elements of the current catalog with another using spatial (RA, DEC) and a list of spectral lines location.

  • Catalog.plot_id is deprecated, in favor of Catalog.plot_symb with label=True.

  • Allow one to use a mask in Catalog.select.

  • Add workaround for reading FITS table written with Catalog, with Astropy 3.0 (because of a bug introduced with their new serialization feature, which does not work with subclasses).

2.4 (24/11/2017)

  • Compatibility with Scipy 1.0 (removal of scipy.stat.threshold).

  • Add compressed FITS files (.fits.fz) to the supported extensions.

Image

  • Add a var option to plot the variance extension with .plot().

Cube

  • Fix bug in cube.spatial_erosion.

Sources

  • Keep the original order of the header keywords in .info().

  • Allow one to set the size of a source without needing the white image.

  • New option to add the white image directly within add_cube.

Catalog

  • Fix unit conversion in Catalog.edgedist.

  • Avoid forcing the ra/dec column names to uppercase.

2.3 (13/09/2017)

  • New function to create a PSF cube in create_psf_cube.

  • Update the mpdaf.drs.rawobj module.

  • New extract_cube_fieldsMap script.

WCS

  • Avoid useless unit conversions in pix2sky and sky2pix.

  • Add back the WCS.rotate method.

Spectrum

  • Fix Spectrum.plot when unit is not angstrom.

  • Add wavelength filtering, thanks to Markus Rexroth (EPFL): wavelet_filter.

Image

  • Fix align_with_image which was modifying the input data.

  • Several bugfixes for Gaussian and Moffat fits.

  • Margin of 1/100th of pixel added in inside.

  • Allow one to set the center outside the parent image in subimage.

Cube

CubeList

  • Avoid warnings with HIERARCH keywords

  • Mask NaNs in the output cube, useful when creating the white-image after.

Sources

  • Fix removal of extension with the optimized source writing. [!87]

  • Add an overwrite parameter to write. [#485]

  • Fix text truncated in source history.

  • New optimal extraction algorithm for “CCD spectroscopy”, Horne, K. 1986.

  • Allow one to set the order for the spline interpolation in add_image.

Catalog

  • Correct bug (naxis inversion) in catalog.select and catalog.edgedist

Pixtable

  • Bugfix for selfcalibrate: make sure that we have enough pixels with a flux in each slice.

v2.2 (24/01/2017)

  • Compatibility with Astropy 1.3

  • Fix direct replacement of .data in a Cube/Image/Spectrum object. [!82]

WCS

  • Fix bugs with the handling of CROTa. [!77]

Image

  • Fix bug in rebin when the factor parameter is a tuple. [#483]

Spectrum

Cube

CubeList

  • Fix offset computation in CubeMosaic, using CRPIX from the output cube.

  • More options in the pycombine methods: MAD, scales, offsets.

Sources

  • Correct behaviour when adding an image not overlapping with Source. [#482]

Catalog

Pixtable

  • A new method selfcalibrate was added to correct the background levels of the slices. This method replaces the subtract_slice_median and divide_slice_median methods (which have been removed). The new method works differently, gives better results, and no more requires to pre-compute a mean sky spectrum with sky_ref. [!78]

v2.1 (16/11/2016)

New Features

  • Allow one to pass optional arguments when opening a FITS file, using the fits_kwargs parameter.

  • Allow one to write CHECKSUM/DATASUM when saving a FITS file (use checksum=True). [!53]

  • Image and Spectrum objects keep now by default the type of the FITS data (like Cube). [!50]

  • Add dtype property to Data classes (Spectrum/Image/Cube).

  • Add WCS naxis1/naxis2 properties which uses naxis from the underlying wcs object.

  • Determine the reference frame from the primary header if possible and don’t force it if not found in the primary header. HST and MUSE files usually have the EQUINOX/RADESYS/RADECSYS keywords only in the primary header, which cause MPDAF to use ICRS instead of FK5. [!47] Add reference frame in WCS.info.

  • Enhance fftconvolve and add this method for Cube. [!52]

  • New method MUSE.get_FSF_from_cube_keywords which creates a cube of FSFs corresponding to the keywords presents in the MUSE data cube primary header.

  • Add small utility function to create field maps.

  • Make zscale available from mpdaf.tools.

  • Move tests and data inside the MPDAF package so that they are installed with MPDAF.

  • Replace nosetest with py.test to run test.

Breaking changes

  • Spectrum methods that return a value of flux or magnitude, return now a tuple (value, error). This breaking change concerns: flux2mag, mean, sum, integrate, abmag_band, abmag_filter_name, abmag_filter.

  • Forbid the use of several (not implemented) methods in CubeMosaic.

  • Remove WCS.set_naxis methods.

WCS

  • Remove WCS.set_naxis methods.

  • Add WCS naxis1/naxis2 properties which uses naxis from the underlying wcs object.

  • Determine the reference frame from the primary header if possible and don’t force it if not found in the primary header. HST and MUSE files usually have the EQUINOX/RADESYS/RADECSYS keywords only in the primary header, which cause mpdaf to use ICRS instead of FK5. Add reference frame in WCS.info.

  • Simplify deg2sexa and sexa2deg.

Data classes (Cube, Image, Spectrum)

  • Enhance reading from an HDUList without having to specify a filename.

  • Image and Spectrum objects keep now by default the type of the FITS data (like Cube).

  • Add dtype property to Data classes (Spectrum/Image/Cube).

  • Make DataArray[item] preserve WCS and/or wavelength info for all legal item selections. Prior to this patch, if c was a cube, c[10] returned an MPDAF Image that didn’t have any WCS information, and c[10:20] returned a Cube without either WCS or wavelength information.

  • Refactor Spectrum/Image/Cube’s methods .convolve and .fftconvolve, with variance propagation.

    In the previous implementation of Image and Spectrum.fftconvolve(), the shape of the ‘other’ array had to match the size of the Image or Spectrum. In the new version, the ‘other’ array can be any size up to the size of the MPDAF object that is being convolved.

    The optional interp argument of Image.fftconvolve() has been removed. Filling masked data and variances with zeros for the duration of the convolution should be sufficient in most cases.

Spectrum

  • Set default limits on the x axis for Spectrum plots.

  • Simplify Spectrum.correlate, Spectrum.fftconvolve_gauss, Spectrum.median_filter and Spectrum._interp.

  • Return flux/magnitude error if relevant.

  • Rewrote Spectrum.resample: When pixel sizes are being increased a decimation filtering stage is now used before regridding, whereas the original behavior was to perform piecewise integrations for each output pixel. When pixel sizes are being reduced, simple linear interpolation is followed by decimation filtering.

Image

  • Fix Image.fwhm which was returning twice the FWHM.

  • Fix bug which caused resample to change the sign of the X-axis increment.

  • Simplify creation of subimages in Image.segment.

  • Reduced memory usage in Image.truncate, Image.regrid, Image.align_with_image. This speeds up align_with_image significantly.

  • Fix exceptions in Image.plot when .wcs is None.

  • Fix bug that sometimes caused Image.plot to fail to show the cursor coordinates of an image.

  • Use zscale from Astropy if available (1.2 and later).

  • Add method .to_ds9() to visualize data in ds9 and interact with it (using pyds9).

Cube

  • Fix bug in Cube.rebin. [!471]

  • Improved the method bandpass_image:

    • If their isn’t a complete overlap between the bandpasses of the filter-curve and the cube, truncate the filter at the edges of the cube instead of raising an exception.

    • When integrating the filter curve over each wavelength channel of the cube, use linear interpolation by default, rather than cubic.

Sources

  • Increase the file reading speed by loading values of dictionaries (spectra, images, cubes and tables) just if necessary.

  • CUBE* keywords became mandatory:

    • CUBE: Name of the MUSE data cube.

    • CUBE_V: Cube version.

  • Some keywords are renamed:

    • ORIGIN -> FROM (Name of the software used to detect the source)

    • ORIGIN_V -> FROM_V (Version of the software used to detect the source)

    • SRC_VERS -> SRC_V (Source version)

    • SOURCE_V -> FORMAT (Version of the mpdaf.sdetect.Source class used to create the source)

    • CONFI -> CONFID (Expert confidence index)

  • Change format of COMMENT and HISTORY

    • COM*** -> COMMENT

    • HIST*** -> HISTORY

    [Date Author] User comment/History

  • Updated Source.info: comments and histories printed more properly.

  • extract_spectra: Add the possibility to extract MOFFAT PSF weighted spectra in addition to the Gaussian PSF.

  • Add primary indexes (with unicity constraint) to mag['BAND'] and z['Z_DESC'] for simpler indexing.

  • Correct behaviour when trying to add image not overlapping with Source [!482].

Catalogs

  • Optimize catalog initialization by not loading all tables.

  • Update the initialization in order to be correct for Numpy arrays and masked arrays.

  • Make Catalog compatible with Python 3.

  • Add comments and histories in catalog generated from a list of Source objects.

  • Update Catalog documentation [!467]

  • Correct issue #466:

    • Raise ValueError if astropy.Table try to convert a string to float/int. The message gives the name of the column.

    • Add warning if a keyword has not the same type in all sources but the conversion is possible.

    • CUBE_V is now a mandatory keyword with the string format.

muselet

  • Changed default SExtractor parameters (QUIET and no segmentation).

  • Little optimization (don’t use mask array for the continuum computation, write NB images with astropy.io.fits, remove RuntimeWarning warnings).

  • muselet now compatible with Python 3.

Pixtable

  • Use a more efficient implementation for PixTable.sky_ref.

  • Allow one to work on PixTable object without the .filename attribute.

  • Fix PixTable.divide_slice_median.

  • Add repr info for PixTable objects.

  • Add unit tests.

v2.0 (05/07/2016)

This is the first public release, as an open source project (BSD licensed). MPDAF is also now compatible with Python 3.3+.

New Features

  • Image.resample: now applies by default an anti-aliasing filter when asked to lower the resolution of an image. The variance is now computed for the re-sampled image.

  • Image.regrid: new method similar to Image.resample, but more flexible in the positioning of the sky in the re-sampled image, and it accepts signed pixel increments. resample now uses regrid and was conserved for compatibility reasons.

  • Image.rotate: now works correctly on images whose pixels have different angular sizes along the X and Y axes.

  • Image.align_with_image: new method to resample the image of the object to give it the same orientation, position, resolution and size as another image.

  • Image.estimate_coordinate_offset: new method which uses a full-image auto-correlation to measure the average offsets between the world coordinates of common astronomical features in two images. This was written primarily to determine coordinate offsets between MUSE images and HST reference images.

  • Image.adjust_coordinates: new method which determines the world coordinate offsets between an image and a reference image, then applies this shift to the reference pixel index of the image.

  • Image.mask_region and Cube.mask_region methods have been revised to support rectangular and square regions that are rotated.

  • Cube.mask_polygon: new method similar to Image.mask_polygon.

  • Cube.bandpass_image: new method for creating images with a given filter response curve.

  • WCS.get_axis_increments: new method which returns the displacements on the sky per pixel increment along the Y and X axes. There is also a new method WCS.set_axis_increments to update the signed displacements on the sky.

  • FieldsMap: new class which reads the possible FIELDMAP extension of the MUSE data cube.

  • FSF: new class which proposes a simple FSF model (Moffat function with a FWHM which varies linearly with the wavelength).

Breaking changes

  • Remove unused methods:

    • Remove the peak detection on a spectrum (peak_detection).

    • Remove interactive plotting methods of Spectrum and Image classes.

    • Remove filter methods on images (minimum_filter, maximum_filter, median_filter) and spectra (median_filter).

    • Remove Image.add_poisson_noise.

    • Remove Image.sum.

    • Remove methods to create images (mask_image, composite_image, make_image).

    • Remove PixTable.get_slices.

    • Remove the SEA script (each step remains available as library).

  • Remove the drs.calibobj module.

  • Remove submodules ZAP and GALPAK.

    • ZAP is now publicly available.

    • GALPAK is also publicly available.

  • The resize methods of Cube, Image and Spectrum have been renamed crop to better indicate their purpose. The new methods are also faster and use less memory, and return the list of slices that have been used to crop the data.

  • The rebin_median methods of Cube, Image and Spectrum have been removed because its effect on the variances couldn’t be computed, and because it didn’t seem scientifically useful.

  • The rebin_factor and rebin_mean methods of Cube, Image and Spectrum have been replace by the rebin method. This method has been simplified to make it more maintainable. The simplification to this function was to effectively truncate the input data to be an integer multiple of the re-binning reduction factor, rather than computing partially sampled output pixels for the edge of the re-binned data.

  • The method Cube.get_lambda has been renamed select_lambda.

  • The method Spectrum.get_lambda has been renamed subspec.

  • The weighting options of Cube.sum has been reimplemented. The old method with the flux conservation has been moved in the SEA library.

  • WCS.get_rot has been corrected to always return the angle between north and the Y axis of the image, in the sense of a rotation of north eastwards of Y. The updated versions of Image.get_rot and Image.rotate resolve this discrepancy, so any software that has been using these functions may see changes.

  • The return value of the get_range methods of Cube, Image and DataArray have been changed to a flat array, rather than an array of two coordinate tuples that only denoted image corners for non-rotated images.

  • WCS.get_naxis1 and WCS.get_naxis2 have been removed, because the underlying WCS.naxis1 and WCS.naxis2 values can be queried directly.

  • WCS.resample and WCS.rotate methods has been removed, because they are no more needed by the equivalent methods in Image.

  • The method Image.add has been removed because it is easy for a user to do the equivalent by using align_with_image() followed by a simple addition of two images.

  • PIL dependency has been removed: now MPDAF doesn’t read .PNG, .JPEG or .BMP images.

WCS

  • When an WCS object is initialized via its cdelt1, cdelt2 and rot parameters, the corresponding coordinate transformation matrix is now calculated in the way recommended in equation 189 of FITS paper II (Calabretta, M. R. & Greisen, E. W. 2002 paper II, A&A, 395, 1077-1122).

  • The WCS initializer now accepts a cd argument, which may be used to set the coordinate transformation directly.

  • set_step now changes the pixel scaling correctly for all FITS files. The previous version, which worked for MUSE FITS files, failed on FITS files whose coordinate transform matrix included any shear terms.

  • A couple of issues have been resolved in get_step. Incorrect values were returned for FITS files with pixels that were rectangular on the sky, rather than square. This didn’t affect typical MUSE FITS files.

  • Don’t write WCS headers with both CDELT1 and CD1_1.

Data classes (Cube, Image, Spectrum)

  • Methods which transform the object return now by default another object. The optional copy argument can be used to request that the input data be transformed in-place.

  • Fix clone to update the NAXIS keywords in the data header

  • Change management of the data type (by default Cube object have the type of the DATA extension)

  • The variance calculation of sqrt has been corrected.

  • Fix Cube.median when used with axis=None.

  • Cube.get_image: add input parameters in the FITS header.

  • Allow one to overwrite BUNIT for data without unit.

  • Fix EXPTIME of combined cubes when cubes overlap (using a median of the exposure map give a more realistic estimate).

  • The mean method of Cube, Image and Spectrum have been rewrote to optionally perform weighted means.

  • In the sum methods of Cube, Image and Spectrum, the weighted mean of the spectral pixels was being multiplied by the total number of input pixels instead of the number of unmasked pixels. This will have resulted in sums that were too small wherever there were masked spectral pixels.

  • Propagate mask in Cube.subcube.

  • Refactor Cube.subcube_circle_aperture to use Cube.subcube and Cube.mask.

  • Make Cube.subcube and Image.subimage as similar as possible. These methods extract rectangular as well as square areas, and they now work correctly for data with rectangular pixels.

  • A couple of problems have been fixed in the code that multiplies an image by a spectrum. The original code didn’t handle variances correctly and crashed due to a non-existent variable.

  • Correct the pixel value shown when moving the mouse over pixels of Image.plot.

Sources

  • Correct bug in Catalog.from_sources

  • Rewrite Catalog.match which was not working properly, now it returns a dictionary.

  • Tell which keyword is missing when creating a source.

  • Allow one to deactivate the masking of invalid values when loading a source. This part (masked_invalid) takes ~40% of the total load time for sources with a lot of tables and columns.

Muselet

  • Optimize matchlines function

  • Split the main function in several functions

  • Remove Numpy warning

  • Use a Numpy array and not a masked array for the weights array

  • Use masked median of inv_variance

  • Make Muselet less verbose when running SExtractor

Pixtable

  • Convert pixtable values to double to avoid precision errors. This avoids discrepancies between _get_pos_sky and _get_pos_sky_numexpr (Numexpr seems to use double precision by default) and probably also with other methods.

  • Fix initial value for subtract_slice_median correction. This value was set to 1 which seems wrong for an additive correction. To emphasize the fact that some combination of slices and quadrants are not valid, and allow one to filter these values later, this commit changes the init value for the count to -1, and the correction to NaN.

  • Fix select_sky in the case of positioned pixel table.

RawFile

v1.2 (13/01/2016)

  • Optimize Cube.subcube and use __getitem__ in subcube/subimage and other methods to speed up things (avoid reading the full cube/image).

  • Add missing units in image methods.

  • Fill data with NaNs only for float arrays, otherwise raise exception.

  • Use a new MpdafUnitsWarning to allow filtering the unit warnings. It can be used this way:

    # filter only MPDAF's warnings
    from mpdaf.tools import MpdafUnitsWarning
    warnings.simplefilter('ignore', category=MpdafUnitsWarning)
    
    # or filter both MPDAF and Astropy warnings
    import astropy.units as u
    warnings.simplefilter('ignore', category=u.UnitsWarning)
    
  • CUNIT FITS keyword: patch to read mum as micron.

  • Correct cube.get_step that returned nothing.

  • Use setuptools for the setup.py:

    • Allow one to use develop mode (python setup.py develop).

    • Install dependencies automatically.

    • Use optional dependencies.

  • Remove unmaintained submodules: quickViz and fsf. quickViz is still available here but maybe not compatible with the latest Aladin version.

  • Remove the displaypixtable module.

  • Avoid a huge memory peak when creating masked arrays with mask=True.

  • Add some tools to print execution times.

  • Added scaling option in Cubelist.combine().

  • Fix cube.var = None to remove the variance part of the Cube.

  • Revert ZAP version to the same as before 1.2b1 (was updated by mistake).

  • Add a new method Image.find_wcs_offsets to find the WCS offset with a reference image.

PixTable

  • Use CRVAL1/CRVAL2 instead of RA/DEC as reference point for positioned pixtables.

  • Remove cos(delta) correction for positioned pixtables.

  • Use directly the binary mask in extract_from_mask.

  • Allow one to use a boolean mask for pixtable selections.

Sources

  • Source.add_image: the order of the rotation is set to 0 in case of an image of 0 and 1.

  • Add methods to manage a history in the sources headers.

  • Use savemask='none' for MASK and SEG extensions.

  • Correct bug in source.write when a column has no unit.

  • Allow one to pass the lambda range and wave unit to Source.extract_spectra.

  • Correct bug in Catalog initialization due to units.

  • Catalog.from_sources: update the default format.

  • Split Source.add_masks in 3 methods: find_sky_mask, find_union_mask and find_intersection_mask.

  • Isolate comments and history in source information.

Muselet

  • Limit the memory usage.

  • Added option to clean detections on skylines.

  • Added exposure map cube.

  • Remove automatic narrow-band images cleaning in muselet.

v1.2b1 (05/11/2015)

Breaking changes

  • Add a new base class for the Cube, Image and Spectrum classes. This allows one to fix some inconsistencies between these classes and to bring more easily new common features.

  • FITS files are now read only when the data is needed: when creating an object the data is not loaded into memory. The data is loaded at the first access of the .data attribute, and the same goes for the variance (and .var). A consequence of these optimization is that the CubeDisk class has been removed.

  • Shape of objects:

    • Remove the shape parameter in constructors. Instead the shape is derived from the datasets.

    • Spectrum’s shape is now a tuple, which is consistent with the Cube and Image classes, and with Numpy arrays.

  • Allow one to specify the data type of Cube/Image/Spectrum in the constructor (and read an extension as an integer array).

  • Change the behavior of the .clone method: now by default it returns an object with the data attribute set to None. This was changed as an optimization, because in most cases (at least in MPDAF’s code) a Numpy array is set to the cloned object, just after the clone, so the Numpy array that was created by clone was discarded. You can get the previous behavior with:

    sp = sptot.clone(data_init=np.zeros)
    

    Or you can set directly a Numpy array to the cloned object:

    sp = sptot.clone()
    sp.data = np.zeros(sptot.shape)
    
  • The fscale attribute of a Cube/Image/Spectrum object has disappeared. MUSE units are now read from the FITS header (it takes into account possible FSCALE keyword). The .unit attribute of Cube/Image/Spectrum saves physical units of the data values and the scale value as an astropy.units object.

  • When a method of MPDAF objects requires a physical value as input, the unit of this value is also given (x=, x_unit=). By default coordinates are in degrees and wavelengths are in angstroms.

  • Results of Source.subcube methods are always centered on the source given in input (columns/row of NaN are added when the source is on the border).

  • Source/Catalog object write and read masked values.

  • From Johan and Benjamin: shell script to interact in Topcat between the muselet catalog and a MUSE datacube opened in ds9.

Changes that should be imperceptible to users

  • Use astropy.wcs for handling the wavelength coordinates.

  • Simplify logging configuration.

  • Cube/Image/Spectrum constructors: allow one to pass a hdulist object in place of the filename (this option should reduce the time when the FITS file is used several times because of the big time spent reading the FITS headers).

v1.1.18.1 (31/07/2015)

  • Full correction of mask_polygon function.

  • Correct a bug in source.show_spec.

  • Source.add_white_image now compute mean(cube) instead of sum(cube).

  • Workaround bug in GCC 5.1 & OpenMP.

  • Add prints for the number of threads in the merging c code.

  • Change redshift table format to have z,zmin,zmax.

  • Use astropy.constants for the c value.

  • Update wcs info method.

  • Correct bug to compute the size of images that are added in source objects.

  • New method Source.add_narrow_band_image_lbdaobs.

  • Default size of 5 arcsec in Source.add_white method.

  • Still have the same type of WCS matrix(CD/PC).

  • Update muselet package to be compatible with new source object.

  • Correct bug in catalog initialization.

v1.1.18 (08/07/2015)

  • Update CubeList.save_combined_cube to be more generic.

  • Optimize C libraries using openmp (cubes combination).

  • Update WCS according to FITS standard.

  • Modify Spectrum.log_plot to be the same as plot with a log stretch.

  • Allow one to create a cube object with a masked array.

  • Correct bug in mask_polygon function of Image object.

  • Possibility to use MAD (median absolute deviation) statistics for sigma-clipping during cube combination.

  • Take into account cos(delta) in source.info.

  • Split mpdaf.logging method in 2 methods (steam_handler/file_handler).

  • Update mask computation of source:

    • option to give a directory containing default files of sextractor.

    • option to remove or not the sextractor files.

    • split add_masks method in two methods (add_seg_images and add_masks).

  • Update source.info method.

  • Correct bug in Cube.aperture.

  • Spectrum extraction code from Jarle (SEA code).

  • Print info in source.add_narrow_band_images().

  • Update Source class: - add_line method. - add_attr/remove_attr methods. - dictionary of tables.

  • Add CubeMosaic class for the merging of a mosaic.

  • Update Source class: - add image rotation in source.info. - rebin mask before applying weight in source.extract_spectra.

  • Initialize a SourceList object from a path name.

  • Image/Cube truncate methods: update computation of boundaries.

  • Correct bug in muselet/setup_files_n.

  • Take into account quadrant in pixtable autocalibration.

  • Fix merged cube headers so that the cube can be ingested by MuseWise.

    • Add needed keywords: RA, DEC, MJD-OBS, DATE-OBS, PI-COI, OBSERVER, OBJECT, ESO INS DROT POSANG, ESO INS MODE

    • Allow one to override OBJECT name

    • Compute a correct EXPTIME for the mosaic case

    • Put the list of merged files in comments, otherwise the keyword value can be too long for MuseWise

  • Update mask computation of source (SEA):

    • take into account rotation of the image

    • replace central detection by detection around the source center.

v1.1.17.1

  • Update CubeList.save_combined_cube to be more generic.

  • Optimize c libraries using openmp.

  • Update WCS according to FITS standard.

  • Modify Spectrum.log_plot to be the same as plot with a log stretch.

  • Allow one to create a cube object with a masked array.

  • Corrected bug in mask_polygon function of Image object.

v1.1.17 (16/06/2015)

  • Correct bug concerning .var attribute of Spectrum/Image/Cube. It must be an array and not a masked array.

  • PixTable: Optimize origin2xoffset and origin2coords

  • Remove tuples in parameters of np.sum/mean/median

  • Update write method of Cube/Image/Spectrum objects

  • Update write method of PixTable

  • Add matplotlib.Axes in plot parameters

  • Update arithmetic methods of Cube/Image in order to accept array as input

  • Add mask_polygon method in image

  • Correct bug in add_mpdaf_method_keywords (MPDAF #365)

  • Make a copy of wcs object during the initialization if Cube/Image/Spectrum objects

  • Update merging of data cubes:

    • method returns a cube object

    • option to compute the variance of the merged cube as the the mean of the variances of the N individual exposures divided by N**2

    • method returns more pixels statistics

  • Source and Catalog classes

  • correct bug in Cube.aperture method

  • Fix numexpr not used when installed.

  • Refactor common part of PixTable.extract

  • Remove ‘ESO PRO’ keywords writing in PixTable. This was changed a long time ago and is not useful anymore.

  • Allow one to extract data from a PixTable with stack numbers.

  • Add a param to PixTable.extract to choose if multiple selection are combined with logical_and (default) or logical_or.

  • Refactor get_* methods of PixTable.

  • Split PixTable.extract in several methods for selecting values. Make a method for each selection type (lambda, slices, ifus, position, …), so that it will be more flexible.

  • Pass units to the extracted PixTable, this avoids muse_exp_combine rejecting pixtables because of different units.

  • Update inputs of fftconvolve_moffat method

  • Add some basic tests for PixTable

  • Refactor PixTable column setters.

  • Correct bug in WCS.__getitem__

  • Add snr option in spectrum.plot to plot data/sqrt(var)

  • ListSource class

  • Update FOCUS detection code to be compatible with new Source object

  • Fixes and enhancements for cubelist:

    • Save MPDAF keywords with comments in the correct order.

    • Save the unit in the output cubes.

    • Fix unit checking, and use the unit/fscale from the first cube if these are not consistent, with a warning.

  • Improve saving of combined cube.

    • FILES list is too long to be both a HIERARCH and CONTINUE keyword. So use a CONTINUE keyword instead.

    • Refactor the saving, and put the saved keywords in the good order.

    • Copy several useful keywords from the source cubes: ORIGIN, TELESCOP, INSTRUME, EQUINOX, RADECSYS, EXPTIME, OBJECT

    • Update EXPTIME, assuming that all files have the same EXPTIME value (to be improved later).

  • Refactor the pixtable extraction from a mask.

  • Subtract_slice_median: don’t correct when all pixels are masked.

  • Change precision in the equality test of two WCSs.

  • Always initialize CubeList.wcs. If there are not equal, just raise a warning.

  • Open raw file without memory mapping

  • Fix flux conservation in rebin methods

  • Cube.subcube method to extract sub-cube

  • Correct Cube.mean

  • Add weights in Cube.sum

  • subtract_slice_median: indent, remove useless stat var, add check for mpdaf_median

  • Add a PixTable.select_stacks method

  • Simplify CubeDisk.truncate

  • Cube.get_image method

  • Cube.subcube_aperture method

  • Corrected median for even-sized tables in merging

  • Source display methods

  • Catalog display methods

  • Correct wcs.info

  • galpak v 1.6.0

  • Spectrum: add gauss_dfit, gauss_asymfit, igauss_asymfit methods

  • Update muselet detection code to be compatible with new Source object

v1.1.16.1

  • Correct bug concerning .var attribute of Spectrum/Image/Cube. It must be an array and not a masked array.

  • PixTable: Optimize origin2xoffset and origin2coords

  • Remove tuples in parameters of np.sum/mean/median

  • Update write method of Cube/Image/Spectrum objects

  • Update write method of PixTable

v1.1.16 (16/03/2015)

  • correct bug in Image.resize method

  • add a script to create a white-light image from a cube

  • correct bug in pixtable.set_lambda method (mpdaf#358)

  • correct bug in pixtable.copy method (mpdaf#359)

  • change method to get the path directory under which mpdaf is installed

  • remove fusion submodule

  • add muselet module

v1.1.15.1 (20/02/2015)

  • Don’t print the msg about Focus each time mpdaf is imported.

  • Don’t load/write the data when only the header must be updated.

  • Add an option to not show the x/y labels in Image.plot

  • Cube merging: Save the list of files that have been merged in the FITS header.

  • Take correctly into account the mask to compute the resulted variance values in cube.sum/mean/median methods.

  • If data are scaled by a constant, variance is scaled by the square of that constant.

  • Correct weight values in least squares fit

  • Replace pyfits by astropy.io.fits in fsf module

v1.1.15 (02/02/2015)

  • update multiprocess methods to be compatible with logger

  • correct bug in Image.mask methods

  • Cube.mask methods

  • Optimize a bit Image.background

  • Update autocalibration methods on pixtable:

    • apply multiplicative correction to stat column

    • PixTableAutoCalib class to store pixtables auto calibration results

  • update cubes merging:

    • cubelist.merging returns cube object

    • cubelist.merging manages BUNIT

  • mpdaf_user.galpak version 1.4.5

  • Spectrum.integrate method

  • Handle float precision in the WCS comparison

  • correct wave unit of pixtable object

  • Source detection package

  • update savemask option in Cube/Image/Spectrum write methods

v1.1.14 (21/01/2015)

  • correct bug in variance computation during CubeDisk.get_white_image method

  • when merging cubes, replace the single sigma clipping parameter into two lower/upper clipping parameters

  • gzip raw file MUSE mask named PAE_July2013.fits

  • restructure C code (tools.c)

  • compute the reference sky spectrum from a pixel table

  • method mask_image that creates a new image from a table of apertures.

  • update Image.mask and Image.mask_ellipse methods

  • allow one to apply a slice on all the cubes of a CubeList.

  • Image/Cube/CubeDisk: correct truncate methods

  • PixTable: new methods to bring all slices to the same median value

    (using sky reference spectrum)

  • update mpdaf logging

  • simplify sky2pix and pix2sky and add a test.

  • replace use of the deprecated commands module with subprocess.

  • update setup.py for MAC

  • add keywords in a FITS header to describe what is done on pixtable

v1.1.13 (17/12/2014)

  • Spectrum/Image/Cube: save mask in DQ extension

  • add setter to pixtable object

  • use numpy methods to convert angles from radians/degrees to degrees/radians

  • add mask_ellipse function in Image object to mask elliptical regions

  • correct bug in world coordinates

  • subtract_slice_median method of PixTable

  • CubeList object to manage merging of cubes

  • pyfits replaced by astropy.io.fits and pywcs replaced by astropy.wcs

  • add inside=T/F parameter for the mask function of Spectrum

v1.1.12 (03/10/2014)

  • the flux scale attribute of Cube/Image/spectrum objects is now never changed by methods.

  • sanity check on wavelength coordinates.

  • new Cube.get_image method that extracts an image from the datacube.

  • write cube/image/spectrum in float32

  • add nearest option for WCS.sky2pix method

  • pixtable: write data/xpos/ypos/lbda column in float32

  • spectrum: oversampling factor for the overplotted Gaussian fit

  • pixtable: code optimization with numexpr

  • zap v0.6

  • galpak v1.1.3

  • correct MOFFAT fit error

v1.1.11 (26/09/2014)

  • Spectrum.GaussFit : update continuum computation

  • Spectrum/Image/Cube - add get_np_data method that returns flux*fscale - add fscale parameter in write methods

  • update docstrings

  • option to overplotted inverse of variance on image

  • Cube.sum/mean methods: mask nan variance values

  • astropy.io.fits.EXTENSION_NAME_CASE_SENSITIVE deprecated -> astropy.io.fits.conf.extension_name_case_sensitive

  • replace “slice” parameter by “sl”

  • add Cube.median and Cube.aperture methods

  • ignore warnings of pyfits.writeto

  • zap v 0.5.1

v1.1.10 (26/08/2014)

  • zap v 0.5.

  • correction of minor bugs in core library

v1.1.9 (31/07/2014)

  • update gitmodules path

  • use astropy to sexa/deg coordinates transforms

  • zap v 0.4.

  • update PixTable documentation

v1.1.8 (09/07/2014)

  • read spheric coordinates of pixel tables.

  • zap v 0.3.

v1.1.7 (26/06/2014)

  • set case sensitive for pixtable extension name.

  • update pixtable coordinates types.

  • correct bug in PixTable.extract method.

  • update pixtable world coordinates.

  • correct PixTable.write method.

  • update documentation of mpdaf installation.

v1.1.6 (02/06/2014)

  • correct error in CalibFile.getImage() method

  • zap update, including the new methods for the offset sky/saturated field case

v1.1.5 (20/04/2014)

  • correct bug in spectrum.write

  • correct bug due to Nan in variance array

  • correct bug in loop_ima

  • support both pyfits and astropy in test_spectrum.py

v1.1.4 (04/02/2014)

  • correct bug in cube.resize method

  • correct typo on right

  • replace print by loggings or errors

  • replace pyfits.setExtensionNameCaseSensitive which is deprecated

  • PEP-8 coding conventions

  • Cube.rebin in the case of naxis < factor

  • autodetect noise extension during Spectrum/Image/Cube creation

  • insert submodule zap

  • replace deprecated methods of pywcs/pyfits replace pywcs by astropy.wcs and pyfits by astropy.fits

  • correct test failures

  • correct bug in Spectrum.fftconvolve_moffat method

  • update wavelength range of Spectrum.rebin() method

  • correct bug in Cube.__getitem__

  • correct bug (typo) in spectrum.write

v1.1.3 (17/01/2014)

  • Image : check if the file exists during the initialization

  • correct bug in the copy of masked array

  • correct bug in cube.rebin_median

  • pixel table visualization

  • fast reconstruction of the white image from RawFile object

  • add check in Spectrum.rebin method

  • correct bug in sub-pixtable extraction

v1.1.2 (11/09/2013)

  • correct coordinates unit in pixtable header

  • pixtable: rename OCS.IPS.PIXSCALE keyword

v1.1.1 (29/08/2013)

  • correct Image.add_poisson method

  • correct bug in PSF module

  • Spectrum/Image/Cube initialization: crval=0 by default and FITS coordinates discarded if wave/wcs is not None

  • Image: fix bug in gaussian fit

  • optimize Image.peak_detection

  • correct bug in WCS.isEqual

  • correct fscale value in multiprocess functions of Cube

  • optimize interactive plots

  • update Channel.get_trimmed_image to do bias subtraction

  • update Image.segment with new parameters

  • add warnings according to M Wendt comments

  • added method to plot a RawFile object

  • added function to reconstruct an image of wavelengths on the detectors from a pixtable

  • output of Image.GaussFit updated for rot=None

  • correct RawFile to have no crash when a SGS extension is present

  • PixTable: multi-extension FITS image format

  • add submodule mpdaf_user.fsf (Camille Parisel/DAHLIA)

v1.1.0 (29/01/2013)

  • mpdaf installation: replace setuptool by distutils

  • add structure (mpdaf_user directory) for user library

  • mpdaf.drs.RawFile: add output detector option

  • mpdaf.drs.CalibFile: add get_image method

  • mpdaf.obj.Spectrum: add normalization in polynomial fit

  • mpdaf.obj.Cube/Image : correct bug to write/load wcs

  • add global parameter CPU for the number of CPUs

  • mpdaf.obj.Cube/Image/Spectrum: correct write methods

  • mpdaf.obj.Spectrum/Image/Cube : rebin_median method rebins cubes/images/spectra using median values.

  • mpdaf.obj.Spectrum : add LSF_convolve method

  • mpdaf.MUSE package that contains tools to manipulate MUSE specific data

  • mpdaf.obj : correct coordinates rebining

  • mpdaf.obj.Image : peaks detection

  • mpdaf.MUSE.LSF : simple MUSE LSF model

  • mpdaf.obj.Cube : multiprocessing on cube iterator

  • mpdaf.obj.Image : update gaussian/moffat fit

  • mpdaf.obj.CubeDisk class to open heavy cube fits file with memory mapping

v1.0.2 (19/11/2012)

  • correct rotation effect in Image.rebin method

  • correct bug in spectrum/Image Gaussian fit

  • remove prettytable package

  • Spectrum/Image/Cube: correct set_item methods

  • method to reconstruct image on the sky from pixtable

  • ima[:,q] or ima[p,:] return Spectrum objects and not 1D images

  • link on new version of HyperFusion

  • Image: add iterative methods for Gaussian and Moffat fit

  • Image: remove matplotlib clear before plotting

  • fusion: update FSF model

  • Spectrum/Image/Cube .primary_header and .data_header attributes

  • fusion: add copy and clean, continue_fit methods

  • pixtable: support new HIERARCH ESO DRS MUSE keywords (MPDAF ticket #23) update HIERARCH ESO PRO MUSE PIXTABLE LIMITS keywords when extracting a pixtable (MPDAF ticket #20)

  • tools: add a Slicer class to convert slices number between various numbering scheme

  • fusion: correct position (cos delta)

  • obj package: correct cos(delta) via pywcs

  • Spectrum: correct variance computation

  • obj package: return np.array in place of list

  • Image: correct variance computation

  • Cube: correct variance computation

  • Cube: add rebin_factor method

  • Image: correct Gauss and Moffat fits (cos delta)

  • Pixtable: correct cos(delta)

  • update documentation

v1.0.1 (27/09/2012)

  • Creation of mpdaf.obj package:

    • Spectrum class manages spectrum object

    • Image class manages image object

    • Cube class manages cube object

  • Creation of mpdaf.obj.coords package:

    • WCS class manages world coordinates in spatial direction (pywcs package is used).

    • WaveCoord class manages world coordinates in spectral direction.

    • deg2sexa and sexa2deg methods transforms coordinates from degree/sexagesimal to sexagesimal/degree.

  • adding selection and arithmetic methods for Spectrum/Image/Cube objects (mpdaf.obj package)

  • complete mpdaf.fusion package (python interface for HyperF-1.0.0)

  • change mpdaf structure to have “import mpdaf”

  • correct bug on memmap file

  • new functionalities for Spectrum object (rebining, filtering, gaussian/polynomial fitting, plotting)

  • documentation

  • bug corrections in Spectrum objects

  • mpdaf.fusion package: link to HyperF_1.0.1

  • add plotting and 2d gaussian fitting for Image objects

  • correct bug to read spectrum/image/cube extensions

  • correct bug in coords.Wave.pixel() method

  • PixTable object:

    • Fix a typo in get_slices output message

    • always read the data from the first exposure

    • use uint32 for origin and dq

  • Image: add functionalities (transform, filter, sub-images)

  • Spectrum/Image/Cube: correct bug for variance initialization

  • Pixtable: optimize and split origin2coords in multiple helpers

  • Update WCS object accoriding to the python notation : (dec,ra)

  • Image: add methods to mask/unmask the image.

  • Update the python interface for HyperF v1.1

  • Add euro3D package

  • Correct error with new version of pywcs (remplace ‘UNITLESS’ by ‘’ for unit type)

  • Compatibility with pyfits 3.0 (The Header.ascardlist() method is deprecated, use the .ascard attribute instead)

  • Pixtable: rewrite the extract function & keep memory map filenames as private attributes

  • Split objs.py in 4 files : spectrum.py, image.py, cube.py, objs.py

  • Pixtable: add a reconstruct_det_image method

  • New release of Spectrum class

  • Create Image from PNG and BMP files

  • Use nosetest for unit tests

  • Add mpdaf.__info__

  • Spectrum/Image/Cube: reorganize copy/in place methods

  • Add Cube iterators

  • Spectrum/Image/Cube: add clone method

  • Add nose and matplotlib as prerequisites

  • obj package: correct fscale use

  • Cube/Image/Spectrum : add mask_selection method

  • Update python interface for HyperFusion v1.2.0

  • Spectrum/Image/Cube: bugs corrections

  • version 1.0.1

v1.0.0 (02/12/2011)

First public release.