--- title: "weatherOz for DPIRD" author: "Rodrigo Pires, Anna Hepworth, Rebecca O'Leary and Adam H. Sparks" output: rmarkdown::html_vignette: toc: true vignette: > %\VignetteIndexEntry{weatherOz for DPIRD} %\VignetteEngine{knitr::rmarkdown_notangle} %\VignetteEncoding{UTF-8} --- ## About DPIRD Data From the DPIRD Weather Website's ["About" Page](https://weather.agric.wa.gov.au/about). > The Department of Primary Industries and Regional Development's (DPIRD) network of automatic weather stations and radars throughout the state provide timely, relevant and local weather data to assist growers and regional communities make more-informed decisions. > > The weather station data includes air temperature, humidity, rainfall, wind speed and direction, with most stations also measuring incoming solar radiation to calculate evaporation. This website includes dashboards for each station to visualise this data. Data from the DPIRD API are licenced under the [Creative Commons Attribution 3.0 Licence (CC BY 3.0 AU)](https://creativecommons.org/licenses/by/3.0/au/deed.en). ## A Note on API Keys All examples in this vignette assume that you have stored your API key in your .Renviron file. See [Chapter 8](https://rstats.wtf/r-startup.html#renviron) in "What They Forgot to Teach You About R" by Bryan _et al._ for more on storing details in your .Renviron if you are unfamiliar. ## Working With DPIRD Data Three functions are provided to streamline fetching data from the DPIRD Weather 2.0 API endpoints. * `get_dpird_extremes()`, which returns the recorded extreme values for the given station in the DPIRD weather station network.; * `get_dpird_minute()`, which returns weather data in minute increments for stations in the DPIRD weather station network with only the past two years being available; and * `get_dpird_summaries()`, which returns weather data in 15 and 30 minute, hourly, daily, monthly or yearly summary values for stations in the DPIRD weather station network. ## Getting Extreme Weather Values The `get_dpird_extremes()` function fetches and returns nicely formatted individual extreme weather summaries from the DPIRD Weather 2.0 API. You must provide a `station_code` and `API_key`, the other arguments, `values` and `include_closed` are optional. ### Available Values for Extreme Weather * all (which will return all of the following values), * erosionCondition, * erosionConditionLast7Days, * erosionConditionLast7DaysDays, * erosionConditionLast7DaysMinutes, * erosionConditionLast14Days, * erosionConditionLast14DaysDays, * erosionConditionLast14DaysMinutes, * erosionConditionMonthToDate, * erosionConditionMonthToDateDays, * erosionConditionMonthToDateMinutes, * erosionConditionMonthToDateStartTime, * erosionConditionSince12AM, * erosionConditionSince12AMMinutes, * erosionConditionSince12AMStartTime, * erosionConditionYearToDate, * erosionConditionYearToDateDays, * erosionConditionYearToDateMinutes, * erosionConditionYearToDateStartTime, * frostCondition, * frostConditionLast7Days, * frostConditionLast7DaysDays, * frostConditionLast7DaysMinutes, * frostConditionLast14Days, * frostConditionLast14DaysDays, * frostConditionLast14DaysMinutes, * frostConditionMonthToDate, * frostConditionMonthToDateDays, * frostConditionMonthToDateMinutes, * frostConditionMonthToDateStartTime, * frostConditionSince9AM, * frostConditionSince9AMMinutes, * frostConditionSince9AMStartTime, * frostConditionTo9AM, * frostConditionTo9AMMinutes, * frostConditionTo9AMStartTime, * frostConditionYearToDate, * frostConditionYearToDate, * frostConditionYearToDateMinutes, * frostConditionYearToDateStartTime, * heatCondition, * heatConditionLast7Days, * heatConditionLast7DaysDays, * heatConditionLast7DaysMinutes, * heatConditionLast14Days, * heatConditionLast14DaysDays, * heatConditionLast14DaysMinutes, * heatConditionMonthToDate, * heatConditionMonthToDateDays, * heatConditionMonthToDateMinutes, * heatConditionMonthToDateStartTime, * heatConditionSince12AM, * heatConditionSince12AMMinutes, * heatConditionSince12AMStartTime, * heatConditionYearToDate, * heatConditionYearToDateDays, * heatConditionYearToDateMinutes, and * heatConditionYearToDateStartTime ### Example 1: Get All Extremes for Northam, WA In the first example, we illustrate how to fetch all extreme values available for Northam. ``` r library(weatherOz) #> Error in library(weatherOz): there is no package called 'weatherOz' (extremes <- get_dpird_extremes( station_code = "NO" )) #> Error in get_dpird_extremes(station_code = "NO"): could not find function "get_dpird_extremes" ``` ### Example 2: Get Selected Extremes for Northam, WA Fetch only soil erosion extreme conditions for Northam, WA. The documentation for `get_dpird_extremes()` contains a full listing of the values that are available to query from this API endpoint. ``` r library(weatherOz) #> Error in library(weatherOz): there is no package called 'weatherOz' ( extremes <- get_dpird_extremes( station_code = "NO", values = "erosionCondition" ) ) #> Error in get_dpird_extremes(station_code = "NO", values = "erosionCondition"): could not find function "get_dpird_extremes" ``` ## Getting Minute Data This function fetches nicely formatted minute weather station data from the DPIRD Weather 2.0 API for a maximum 24-hour period. You must provide a `station_code` and `API_key`, the other arguments, `start_date_time`, `minutes` and `values` are optional. ### Available Values for Minute Data * all (which will return all of the following values), * airTemperature, * dateTime, * dewPoint, * rainfall, * relativeHumidity, * soilTemperature, * solarIrradiance, * wetBulb, * wind, * windAvgSpeed, * windMaxSpeed, and * windMinSpeed ### Example 3: Get All Minute Data for the Past 24 Hours ``` r library(weatherOz) #> Error in library(weatherOz): there is no package called 'weatherOz' ( min_dat <- get_dpird_minute( station_code = "NO" ) ) #> Error in get_dpird_minute(station_code = "NO"): could not find function "get_dpird_minute" ``` ### Example 4: Get Specific Time and Date Data for Specific Values If you wish to supply a specific start date and time and values, you may do so as shown here. ``` r library(weatherOz) #> Error in library(weatherOz): there is no package called 'weatherOz' ( min_dat_t_rad_wind <- get_dpird_minute( station_code = "NO", start_date_time = "2023-02-01 13:00:00", minutes = 1440, values = c("airTemperature", "solarIrradiance", "wind") ) ) #> Error in get_dpird_minute(station_code = "NO", start_date_time = "2023-02-01 13:00:00", : could not find function "get_dpird_minute" ``` ## Getting Summary Data The function, `get_dpird_summary()`, fetches nicely formatted minute weather station data from the DPIRD Weather 2.0 API for a maximum 24-hour period. You must provide a `station_code` and `API_key`, the other arguments, `start_date_time`, `minutes` and `values` are optional. ### Available Values for Summary Data * all (which will return all of the following values), * airTemperature, * airTemperatureAvg, * airTemperatureMax, * airTemperatureMaxTime, * airTemperatureMin, * airTemperatureMinTime, * apparentAirTemperature, * apparentAirTemperatureAvg, * apparentAirTemperatureMax, * apparentAirTemperatureMaxTime, * apparentAirTemperatureMin, * apparentAirTemperatureMinTime, * barometricPressure, * barometricPressureAvg, * barometricPressureMax, * barometricPressureMaxTime, * barometricPressureMin, * barometricPressureMinTime, * battery, * batteryMinVoltage, * batteryMinVoltageDateTime, * chillHours, * deltaT, * deltaTAvg, * deltaTMax, * deltaTMaxTime, * deltaTMin, * deltaTMinTime, * dewPoint, * dewPointAvg, * dewPointMax, * dewPointMaxTime, * dewPointMin, * dewPointMinTime, * erosionCondition, * erosionConditionMinutes, * erosionConditionStartTime, * errors, * etoShortCrop, * etoTallCrop, * evapotranspiration, * frostCondition, * frostConditionMinutes, * frostConditionStartTime, * heatCondition, * heatConditionMinutes, * heatConditionStartTime, * observations, * observationsCount, * observationsPercentage, * panEvaporation, * rainfall, * relativeHumidity, * relativeHumidityAvg, * relativeHumidityMax, * relativeHumidityMaxTime, * relativeHumidityMin, * relativeHumidityMinTime, * richardsonUnits, * soilTemperature, * soilTemperatureAvg, * soilTemperatureMax, * soilTemperatureMaxTime, * soilTemperatureMin, * soilTemperatureMinTime, * solarExposure, * wetBulb, * wetBulbAvg, * wetBulbMax, * wetBulbMaxTime, * wetBulbMin, * wetBulbMinTime, * wind, * windAvgSpeed, and * windMaxSpeed ### What You Get Back This function returns a `data.table` with `station_code` and the date interval queried together with the requested weather variables in alphabetical order. Please note this function converts date-time columns from Coordinated Universal Time 'UTC'} to Australian Western Standard Time 'AWST'. The first ten columns will always be: * `station_code`, * `station_name`, * `longitude`, * `latitude`, * `year`, * `month`, * `day`, * `hour`, * `minute`, and if `month` or finer is present, * `date` (a combination of year, month, day, hour, minute as appropriate) ### Example 5: Get Annual Rainfall Since 2017 Use the default value for end date (current system date) to get annual rainfall since 2017 until current year for Capel. ``` r library(weatherOz) #> Error in library(weatherOz): there is no package called 'weatherOz' ( annual_rain <- get_dpird_summaries( station_code = "CL001", start_date = "20170101", interval = "yearly", values = "rainfall" ) ) #> Error in get_dpird_summaries(station_code = "CL001", start_date = "20170101", : could not find function "get_dpird_summaries" ``` ### Example 6: Get Monthly Rainfall Since 2017 Use the default value for end date (current system date) to get monthly rainfall since 2017 until current year for Capel. ``` r library(weatherOz) #> Error in library(weatherOz): there is no package called 'weatherOz' ( monthly_rain <- get_dpird_summaries( station_code = "CL001", start_date = "20170101", interval = "monthly", values = "rainfall" ) ) #> Error in get_dpird_summaries(station_code = "CL001", start_date = "20170101", : could not find function "get_dpird_summaries" ``` ### Example 7: Get Daily Rainfall and Wind From Beginning of 2017 to End of 2018 Use the default value for end date (current system date) to get daily rainfall and wind records from 2017-01-01 to 2018-12-31 for Binnu. Note that the Binnu station has two wind heights, 3m and 10m. ``` r library(weatherOz) #> Error in library(weatherOz): there is no package called 'weatherOz' ( daily_wind_rain <- get_dpird_summaries( station_code = "BI", start_date = "20170101", end_date = "2018-12-31", interval = "daily", values = c("rainfall", "wind") ) ) #> Error in get_dpird_summaries(station_code = "BI", start_date = "20170101", : could not find function "get_dpird_summaries" ``` ### Example 8: Get Hourly Rainfall and Wind From Beginning of 2022 to Current Use the default value for end date (current system date) to get hourly rainfall and wind records from 2022-01-01 to Current Date for Binnu. Note that the Binnu station has two wind heights, 3m and 10m. ``` r library(weatherOz) #> Error in library(weatherOz): there is no package called 'weatherOz' ( hourly_wind_rain <- get_dpird_summaries( station_code = "BI", start_date = "20220101", interval = "hourly", values = c("rainfall", "wind") ) ) #> Error in get_dpird_summaries(station_code = "BI", start_date = "20220101", : could not find function "get_dpird_summaries" ``` ## Getting APSIM-ready Data For work with APSIM, you can use `get_dpird_apsim()` to get an object of DPIRD weather data in your R session that's ready for saving using `write_apsim_met()`, which is re-exported from the CRAN package [apsimx] for your convenience. This function only needs the `station_code`, `start_date`, `end_date` and your `api_key` values to return the necessary values. ### What You Get Back An object of {apsimx} 'met' class, compatible with a `data.frame`, that has daily data that include year, day, radiation, max temperature, min temperature, rainfall, relative humidity, evaporation and windspeed. ### Example 9: Get APSIM Formatted Data for Binnu From 2022-04-01 to 2022-11-01 ``` r library(weatherOz) #> Error in library(weatherOz): there is no package called 'weatherOz' ( binnu <- get_dpird_apsim( station_code = "BI", start_date = "20220101", end_date = "20221231" ) ) #> Error in get_dpird_apsim(station_code = "BI", start_date = "20220101", : could not find function "get_dpird_apsim" ``` ## Working With DPIRD Metadata Three functions are provided to assist in fetching metadata about the stations. * `find_nearby_stations()`, which returns a `data.table` with the nearest weather stations to a given geographic point or known station in either the DPIRD or BOM (from SILO) networks. * `find_stations_in()`, which returns a `data.table` with the weather stations falling within a given geographic area in either the DPIRD or BOM (from SILO) networks. * `get_dpird_availability()`, which returns a `data.table` with the availability for weather stations in the DPIRD network providing the up time and data availability for a given period of time. * `get_stations_metadata()`, which returns a `data.table` with the latest and most up-to-date information available from the Weather 2.0 API on the stations' geographic locations, hardware details, *e.g.,* wind mast height, and recording capabilities. ### Finding Nearby Stations ### Example 10: Finding Stations Nearby a Known Station Query WA only stations and return DPIRD's stations nearest to the Northam, WA station, "NO", returning stations with 50 km of this station. ``` r library(weatherOz) #> Error in library(weatherOz): there is no package called 'weatherOz' ( wa_stn <- find_nearby_stations( station_code = "NO", distance_km = 50, which_api = "dpird" ) ) #> Error in find_nearby_stations(station_code = "NO", distance_km = 50, which_api = "dpird"): could not find function "find_nearby_stations" ``` ### Example 11: Finding Stations Nearby a Given Longitude and Latitude Using the longitude and latitude for Northam, WA, find all DPIRD stations within a 50km radius of this geographic point. ``` r library(weatherOz) #> Error in library(weatherOz): there is no package called 'weatherOz' ( wa_stn_lonlat <- find_nearby_stations( longitude = 116.6620, latitude = -31.6540, distance_km = 50, which_api = "dpird" ) ) #> Error in find_nearby_stations(longitude = 116.662, latitude = -31.654, : could not find function "find_nearby_stations" ``` ### Example 12: Finding Stations in Both the DPIRD and SILO Data Sets Query stations nearest DPIRD's Northam, WA station, "NO" and return both DPIRD and SILO/BOM stations within 50 km of this station. ``` r library(weatherOz) #> Error in library(weatherOz): there is no package called 'weatherOz' ( wa_stn_all <- find_nearby_stations( station_code = "NO", distance_km = 50, which_api = "all" ) ) #> Error in find_nearby_stations(station_code = "NO", distance_km = 50, which_api = "all"): could not find function "find_nearby_stations" ``` ### Example 13: Finding Stations in the Southwest Agriculture Region of Western Australia Using `find_stations_in()` is different than `find_nearby_stations()` as it finds any stations that fall within a boundary that you provide rather than using a single point to search from. For detailed examples using named places or bounding boxes, see the "weatherOz for SILO" vignette. The {sf} object, `south_west_agricultural_region`, is provided with {weatherOz} under the [CC BY 4.0 Licence](https://creativecommons.org/licenses/by/4.0/deed.en) from the Department of Primary Industries and Regional Development (DPIRD), so we can extract stations within this area of Western Australia. First, we can plot the `south_west_agricultural_region` to see what it looks like. ``` r library(weatherOz) #> Error in library(weatherOz): there is no package called 'weatherOz' library(ggplot2) library(ggthemes) library(sf) #> Linking to GEOS 3.11.0, GDAL 3.5.3, PROJ 9.1.0; sf_use_s2() is TRUE ggplot(south_west_agricultural_region) + geom_sf() + theme_map() #> Error in eval(expr, envir, enclos): object 'south_west_agricultural_region' not found ``` Now we can use that to find stations that fall only within that part of Western Australia. We'll use the coordinate reference system (CRS) provided by this {sf} object and find all stations, including those that have closed. ``` r sw_wa <- find_stations_in( x = south_west_agricultural_region, include_closed = TRUE, crs = sf::st_crs(south_west_agricultural_region) ) #> Error in find_stations_in(x = south_west_agricultural_region, include_closed = TRUE, : could not find function "find_stations_in" sw_wa #> Error in eval(expr, envir, enclos): object 'sw_wa' not found ``` We need to convert the `sw_wa` object from a `data.table` to an `sf` object and transform it to use the same CRS as the `south_west_agricultural_region` object to map the results. ``` r sw_wa <- st_as_sf( x = sw_wa, coords = c("longitude", "latitude"), crs = "EPSG:4326" ) #> Error in eval(expr, envir, enclos): object 'sw_wa' not found sw_wa <- st_transform(x = sw_wa, crs = st_crs(south_west_agricultural_region)) #> Error in eval(expr, envir, enclos): object 'sw_wa' not found ``` Now we can use {ggplot2} to plot the stations indicating whether they are still open or they are closed. ``` r ggplot(south_west_agricultural_region) + geom_sf(fill = "white") + geom_sf(data = sw_wa, alpha = 0.65, size = 2, aes(colour = status)) + theme_map() #> Error in eval(expr, envir, enclos): object 'south_west_agricultural_region' not found ``` ### Checking Station Uptime or Availability ### Example 14: Checking Station Availability for Current Year Check the availability of the Westonia station since the start of the current year using the default functionality with no `start_date` or `end_date`. ``` r library(weatherOz) #> Error in library(weatherOz): there is no package called 'weatherOz' (WS001 <- get_dpird_availability( station_code = "WS001" )) #> Error in get_dpird_availability(station_code = "WS001"): could not find function "get_dpird_availability" ``` ### Example 15: Checking Station Availability for a Set Time Period Check the availability of the Binnu station for January of 2018. When a custom `start_date` is provided an `end_date` must also be provided. ``` r library(weatherOz) #> Error in library(weatherOz): there is no package called 'weatherOz' ( BI_201801 <- get_dpird_availability( station_code = "BI", start_date = "2018-01-01", end_date = "2018-01-31" ) ) #> Error in get_dpird_availability(station_code = "BI", start_date = "2018-01-01", : could not find function "get_dpird_availability" ``` ### Getting Station Metadata for the DPIRD Network Stations The `get_stations_metadata()` function is shared with the SILO functions as well, so this function will retrieve data from both weather APIs. Shown here is how to use it for DPIRD data only and with an example of DPIRD specific information, namely including closed stations and rich metadata. ### Example 16: Get DPIRD Station Metadata The `get_stations_metadata()` function allows you to get details about the stations themselves for stations in the DPIRD and SILO (BOM) networks in one function. Here we demonstrate how to get the metadata for the DPIRD stations only. ``` r library(weatherOz) #> Error in library(weatherOz): there is no package called 'weatherOz' (metadata <- get_stations_metadata(which_api = "dpird")) #> Error in get_stations_metadata(which_api = "dpird"): could not find function "get_stations_metadata" ``` ### Example 17: Get Rich DPIRD Station Metadata and Include Closed Stations You can fetch additional information about the DPIRD stations as well as getting data for stations that are no longer open like so with the `rich` and `include_closed` arguments set to `TRUE`. ``` r library(weatherOz) #> Error in library(weatherOz): there is no package called 'weatherOz' (metadata <- get_stations_metadata(which_api = "dpird", include_closed = TRUE, rich = TRUE)) #> Error in get_stations_metadata(which_api = "dpird", include_closed = TRUE, : could not find function "get_stations_metadata" ```