Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add readAll() for SpatRaster and SpatRasterDataset #1660

Merged
merged 2 commits into from
Dec 10, 2024

Conversation

dfriend21
Copy link
Contributor

This pull request adds a readAll function for both SpatRaster and SpatRasterDataset that reads all values into memory. set.values() already accomplishes this for a SpatRaster. However, there isn't an equivalent function for a SpatRasterDataset, so I created a function to do this.

It didn't make sense to create another S4 version of set.values() for SpatRasterDataset since the function would only be for reading values into memory, not for setting values. Because of this, I created a readAll function, and since I was making one for SpatRasterDataset I also made one for SpatRaster. Part of the reason for this is that I feel like it'd be nice to have a dedicated function for reading in the values whose name more clearly indicates it's purpose, since it isn't obvious from the name set.values that the function reads in values from memory (related: #507).

Here's an example - it's also a good example of my usual use-case for explicitly reading values into memory.

library(terra)

# create the URLs for reading in the rasters from the WCS
layers <- c("litter", "shrub")
years <- c(1985, 2023)
template <- c("WCS:https://www.mrlc.gov/geoserver/rcmap_%s/wcs?coverage=rcmap_%s__rcmap_%s_%s")
paths <- lapply(layers, function(layer_i) {
  sprintf(template, layer_i, layer_i, layer_i, years)
})

# define our area of interest
e <- ext(-1631650, -1626603, 1514452, 1517650)

# read in a single layer and read values in with 'readAll'
r <- rast(paths[[1]][1], win = e)
inMemory(r)
readAll(r)
inMemory(r)

# read in the all the layers and convert to a SpatRasterDataset
lst <- lapply(paths, rast, win = e)
sds <- sds(lst)

# use readAll to read all values into memory
sapply(sds, inMemory)
readAll(sds)
sapply(sds, inMemory)

plot(rast(sds))

@rhijmans rhijmans merged commit 64411c5 into rspatial:master Dec 10, 2024
4 checks passed
@rhijmans
Copy link
Member

Thank you, that looks good. I think I wanted to hide this kind of method to avoid malpractice, but I am sure there are good use-cases in which readAll could speed up the analysis.

rhijmans added a commit that referenced this pull request Dec 10, 2024
@rhijmans
Copy link
Member

I renamed terra::readAll to to terra::toMemory because raster::readAll uses "object" in stead of "x" so that gave a conflict that is a bit tricky to resolve; and to better separate it from readStart/Values/Stop.

I changed the methods so that they do not change the objects in-place (and added set.values<SpatRasterDataset> for if you want to do that. Only set.xxx should change objects in place (the problem is that all possible copies of the object are also changed).

netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Jan 5, 2025
# version 1.8-5

## bug fixes

- `Spatsample(method='stratified', ext=e)` returned the wrong sampling
  coordinates [#1628](rspatial/terra#1628)
  by Barnabas Harris

- `spatSample(method='stratified')` could fail with small sample sizes
  [#1503](rspatial/terra#1503) by karluf

- transparency (alpha) did not work with RGB
  plotting. [#1642](rspatial/terra#1642) by
  Timothée Giraud

- rasterization failed on very large rasters
  [#1636](rspatial/terra#1636) by Mary
  Fisher, [#1463](rspatial/terra#1463) by
  Nic Spono and [#1281](rspatial/terra#1281)
  by Sebastian Dunnett

- `tmpFiles` only looked in the default temp files folder
  [#1630](rspatial/terra#1630) by
  smckenzie1986

- `where.min` did not work well if there were negative values
  [#1634](rspatial/terra#1634) by Michael
  Sumner

- `plet<SpatRaster>` now works for RGB rasters and rasters with a
  color table [#1596](rspatial/terra#1596)
  by Agustin Lobo

- `vect<MULTIPOINT WKT>` did not work properly
  [#1376](rspatial/terra#1376) by
  silasprincipe

- `compareGeom<SpatVector>` did not work
  [#1654](rspatial/terra#1654) by Jason
  Flower

- `buffer<SpatVector>` is now more accurate buffers for lonlat
  polygons [#1616](rspatial/terra#1616) by
  Roberto Amaral-Santos

- `terra:interpNear` used square windows, not circles, beyond 100
  points [#1509](rspatial/terra#1509) by
  Jean-Luc Dupouey

- `vect` read INT64 fields as integers, sometimes leading to
  overflows. [#1666](rspatial/terra#1666) by
  bengannon-fc

- `plot` showed a legend title even if none was requestd if title
  parameters were specified
  . [#1664](rspatial/terra#1664) by Márcia
  Barbosa



## enhancements

n- improved documentation of `writeVector` overwrite when using
  layers. [#1573](rspatial/terra#1573) by
  Todd West

- improved treatment of (supposedly) flipped rasters by Timothée
  Giraud [#1627](rspatial/terra#1627) and
  fchianucci [#1646](rspatial/terra#1646)

- added `map.pal("random")`
  [#1631](rspatial/terra#1631) by Agustin
  Lobo

- expressions can now be used in legend titles
  [#1626](rspatial/terra#1626) by Noah
  Goodkind

- `app` and `tapp` now emit a warning when factors are coerced to
  numeric [#1566](rspatial/terra#1566) by
  shuysman

- `plet<SpatRaster>` now has argument "stretch" for RGB rasters
  [#1596](rspatial/terra#1596) by Agustin

- `%%` and `%/%` now behave the same for SpatRaster as for (base R)
  numbers [#1661](rspatial/terra#1661) by
  Klaus Huebert

## new

- `patches` with option `valus=TRUE` can now distinguish regions based
  on their cell values (instead of only NA vs not-NA)
  [#495](rspatial/terra#495) by Jakub
  Nowosad and [#1632](rspatial/terra#1632)
  by Agustin Lobo

- `rowSums`, `rowMeans`, `colSums` and `colMeans` for SpatRaster

- `metags` for SpatRasterDataset
  [#1624](rspatial/terra#1624) by Andrea
  Manica

- `metags` for layers (bands) of SpatRaster are now saved to and read
  from GTiff files
  [#1071](rspatial/terra#1071) by Mike
  Koontz

- `global` has new effcient functions "anyNA" and "anynotNA"
  [#1540](rspatial/terra#1540) by Kevin J
  Wolz

- `wrap`, `saveRDS` and `serialize` for
  SpatExtent. [#1430](rspatial/terra#1430)
  by BastienFR

- `vect<SpatGraticule>` method suggested in relation to [tidyterra
  #155](dieghernan/tidyterra#155) by Diego
  Hernangómez

- `toMemory<SpatRaster>` and `<SpatRasterDataset>` methods
  [#1660](rspatial/terra#1660) by Derek Friend
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants