Writing NetCDF4 Data using Python (codes included)

Utpal Kumar   7 minute read      

Loading likes…

Suppose we have temperature values on a latitude-longitude grid at several times. Saving only the array leaves important questions unanswered: Which axis is time? Where are the grid points? What units do the values use? A NetCDF file lets us save those descriptions alongside the numbers.

Here, we will create the small file used in the companion reading NetCDF tutorial. It contains three time steps on a five-by-six grid, with one deliberately missing value. The temperatures are synthetic teaching data, not observations or model output. Keeping the example small makes it easier to inspect everything we write.

Download the scripts and set up Python

Save the complete writing script and reading script in a new working directory. On macOS or Linux:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install netCDF4 numpy
python write_netcdf.py
python read_netcdf.py example.nc

On Windows, use py -m venv .venv and activate the environment in PowerShell with .venv\Scripts\Activate.ps1. Conda users can instead install with conda install -c conda-forge netcdf4 numpy.

The writing script creates example.nc in the current directory and refuses to overwrite an existing file. If you already downloaded the sample from the reading post, use a different output name, such as python write_netcdf.py my-example.nc, or work in a fresh directory.

Plan the file before writing it

Our file will contain three dimensions: time, lat, and lon. The coordinate variables use those same names. The main variable, temperature, has dimension order (time, lat, lon).

This distinction matters. A dimension records the axis length, while a coordinate variable records the values along that axis. Creating a dimension named time does not automatically save any dates.

We use the NETCDF4 format. The similarly named NETCDF4_CLASSIC format also uses HDF5 storage, but restricts the data model to classic features. It is not the old NetCDF-3 file format. If another program must read your output, check which formats it supports before choosing one. Unidata’s format overview explains these distinctions.

Prepare the arrays

The following creates the data used by the downloadable script:

from datetime import datetime, timedelta
import numpy as np
from netCDF4 import Dataset, date2num

latitudes = np.linspace(35.0, 39.0, 5, dtype="f4")
longitudes = np.linspace(-124.0, -119.0, 6, dtype="f4")
dates = [datetime(2026, 1, 1) + timedelta(hours=6 * i)
         for i in range(3)]

temperature = (
    280.0
    + 0.5 * np.arange(3)[:, None, None]
    + 0.2 * (latitudes[None, :, None] - 35.0)
    + 0.1 * (longitudes[None, None, :] + 124.0)
).astype("f4")
temperature = np.ma.array(temperature, mask=False)
temperature[0, 0, 0] = np.ma.masked

The formula simply makes a smooth field that changes predictably with time and position. Its shape is (3, 5, 6). The first value is masked so that we can check missing-data handling after saving and reopening the file. In your own work, this is where you would supply your measured or modeled array.

The times are January 1, 2026 at 00:00, 06:00, and 12:00 UTC. Explicit dates make the example repeatable; they do not depend on the day you run it.

Create dimensions and variables, then write the data

Run this block after preparing the arrays above, or simply run the downloaded script. Keep everything that writes through ds inside the with block:

with Dataset("example.nc", "w", format="NETCDF4",
             clobber=False) as ds:
    ds.createDimension("time", None)
    ds.createDimension("lat", len(latitudes))
    ds.createDimension("lon", len(longitudes))

    ds.title = "Synthetic temperature example"
    ds.source = "Deterministic teaching data; not observations or model output"

    lat = ds.createVariable("lat", "f4", ("lat",))
    lat.standard_name = "latitude"
    lat.units = "degrees_north"
    lat.axis = "Y"

    lon = ds.createVariable("lon", "f4", ("lon",))
    lon.standard_name = "longitude"
    lon.units = "degrees_east"
    lon.axis = "X"

    time = ds.createVariable("time", "f8", ("time",))
    time.standard_name = "time"
    time.units = "hours since 2026-01-01 00:00:00"
    time.calendar = "standard"
    time.axis = "T"

    temp = ds.createVariable(
        "temperature", "f4", ("time", "lat", "lon"),
        fill_value=-9999.0, compression="zlib", complevel=4,
    )
    temp.standard_name = "air_temperature"
    temp.long_name = "Synthetic air temperature"
    temp.units = "K"

    lat[:] = latitudes
    lon[:] = longitudes
    time[:] = date2num(dates, units=time.units,
                      calendar=time.calendar)
    temp[:] = temperature

There are several details worth checking before adapting this to your data.

First, None makes the time dimension unlimited, so later records can extend it. It starts at length zero and grows when we write the arrays. The latitude and longitude dimensions have fixed lengths. NetCDF-4 permits multiple unlimited dimensions, although this example needs only one.

Second, the dimension tuple on temperature is the array order. An array with shape (lat, lon, time) cannot be assigned as though it were (time, lat, lon). Do not fix a shape error by reshaping blindly; first establish what each axis means.

Third, f4 stores 32-bit floating-point values and f8 stores 64-bit values. We use f4 for this simple temperature field and f8 for time. Choose precision to suit the scientific data rather than assuming every variable needs the largest type.

The context manager closes the file when the block ends. clobber=False protects an existing output from being replaced. Without that protection, opening a file in write mode can overwrite it. See the netCDF4-Python API for these creation options.

Store useful metadata and actual time values

The global title and source attributes describe the whole file. Variable attributes explain individual arrays. For example, temperature.units = "K" makes the stored temperature unit explicit, while standard_name identifies the physical quantity using an established name.

The coordinate units and calendar follow conventions described by the CF community. These help other tools interpret the data. Adding a few recognizable attributes alone does not prove that a production dataset is fully CF-compliant; check the applicable requirements before declaring compliance.

Notice the assignment to time[:]. Calling date2num only calculates numeric times; it does not write them into the file. Here we both convert the dates and assign the result. The stored values are 0, 6, and 12, and the units attribute supplies their reference date. Every temperature record should have its corresponding time value.

Missing values and compression

We set the fill value when creating the temperature variable. The masked element is written using that value, and the default netCDF4 reader masks it again when reading. Choose a fill value outside the valid data range. It is a marker for missing data, not a physical temperature to include in a mean.

The example also enables zlib compression at level four. This compression is lossless: it does not deliberately round the stored values. The earlier conversion to 32-bit floats is a separate precision choice. Compression may not reduce the total size of a tiny file because metadata has overhead; its usefulness becomes clearer with larger arrays.

For large files, access patterns matter too. Reading one map at a time and reading one location through many times may favor different chunk layouts. Start with the library defaults and measure representative reads before tuning chunk sizes. The netCDF4 compression guidance describes the available options.

Reopen the file and check it

A successful write should be followed by a read. This short check verifies dimensions, the missing cell, and the saved dates:

from netCDF4 import Dataset, num2date
import numpy as np

with Dataset("example.nc") as ds:
    temp = ds.variables["temperature"]
    time = ds.variables["time"]
    assert temp.dimensions == ("time", "lat", "lon")
    assert temp.shape == (3, 5, 6)
    assert np.ma.getmaskarray(temp[:]).sum() == 1
    assert ds.dimensions["time"].isunlimited()
    decoded = num2date(time[:], time.units, calendar=time.calendar)
    print([date.strftime("%Y-%m-%d %H:%M:%S") for date in decoded])

The full reading script also prints metadata and extracts a point time series. Running it on your newly written file is a useful check that the two sides of the workflow agree.

Append another time step safely

An unlimited time dimension lets us add a record without recreating the dataset. To try this without modifying the original, download append_netcdf.py and run:

python append_netcdf.py example.nc example-appended.nc
python read_netcdf.py example-appended.nc

The script first copies the original file and refuses to overwrite an existing destination. It opens the copy in r+ mode, which modifies an existing file. It then appends a synthetic field at 18:00 UTC, half a kelvin warmer than the preceding field, and writes both the new data and its timestamp at the same record index. The temperature shape becomes (4, 5, 6); example.nc stays unchanged.

For real data, check the new field’s shape, units, coordinates, and timestamp before writing. Appending a temperature slice without its time coordinate creates an incomplete record. A backup is also sensible when modifying a valuable file, since a failed write is not automatically rolled back.

Further reading

KEEP THE CURIOSITY GOING

Found this article useful?

A little appreciation goes a long way.

Loading likes…

Disclaimer of liability

The information provided by the Earth Inversion is made available for educational purposes only.

Whilst we endeavor to keep the information up-to-date and correct. Earth Inversion makes no representations or warranties of any kind, express or implied about the completeness, accuracy, reliability, suitability or availability with respect to the website or the information, products, services or related graphics content on the website for any purpose.

UNDER NO CIRCUMSTANCE SHALL WE HAVE ANY LIABILITY TO YOU FOR ANY LOSS OR DAMAGE OF ANY KIND INCURRED AS A RESULT OF THE USE OF THE SITE OR RELIANCE ON ANY INFORMATION PROVIDED ON THE SITE. ANY RELIANCE YOU PLACED ON SUCH MATERIAL IS THEREFORE STRICTLY AT YOUR OWN RISK.


Leave a comment