get_SDA_property() all methods now support
miscellaneous_areas argument. This defaults to
FALSE for the methods it was previously implemented for–so
be aware that queries using "Dominant Component" or
"Dominant Condition" (which previously did not respond to
miscellaneous_areas) may have the number of rows in result
reduced due to omission of miscellaneous land types. If this is
unexpected or undesired, please use
miscellaneous_areas=TRUE.
(https://github.com/ncss-tech/soilDB/issues/257)
Adds get_NASIS_metadata() and helper method
get_NASIS_column_metadata() and other new tools for working
with uncode(), factors and NASIS metadata cached in the
package.
Bug fix for get_cosoilmoist_from_SDA(); thanks to
Monika Shea @monikashea for reporting the problem
(https://github.com/ncss-tech/soilDB/issues/253)
fetchNASIS_report() now works with the output from
"get_site_from_NASIS" report (useful for site records
without associated pedons)
createSSURGO() gains arguments quiet
and include_spatial to toggle messages and inclusion of
spatial data in created SQLite database, respectively
downloadSSURGO() now handles queries against
sacatalog that return no results more gracefully
get_OSD(): Add fix_ocr_errors argument
for result='json' to fix common optical character
recognition errors associated with horizon designations and colors
(NOTE: does NOT fix depths)
fetchSCAN() gains timeseries argument
to support batch downloads of hourly data. Additional arguments
(...) are passed as a raw request, allowing other
parameters sent in request to be modified if needed, and bypassing batch
functionality. This is similar to the req argument syntax
used in earlier versions of this function, which had been deprecated for
some time now. Thanks to Matthew Morriss for raising the discussion item
@morrismc99
(https://github.com/ncss-tech/soilDB/discussions/260).
{curl} moved from Suggests to Imports;
curl::curl_download() is now used instead of
utils::download.file() because it seems to handle SSL
certificates better on some networks.
fetchNASIS("components'): Fix and refactor (
breaking change from 2.7.1, which introduced a
bug/inconsistency) of recent change; duplicates argument is
now required to merge in data from mapunit/legend tables (where
many:1 relationships between legend/mapunit and datamapunit are
possible). In 2.7.1 possibly incomplete mapunit/legend tables could be
joined to SoilProfileCollection result (if and only if the tables were
populated in selected set/local DB). Does not change historic
(<=2.7.0) default behavior. Thanks to @dylanbeaudette for suggesting use of
get_component_correlation_data_from_NASIS_db()
here.
Several fixes for Roxygen documentation (notably for
ROSETTA() and various NASIS web report related functions)
that were missing @export tags. Several previously-exported
functions missed being explicitly exported in the new Roxygen-generate
NAMESPACE. These unintentional omissions from 2.7.1 NAMESPACE have been
resolved.
Fixes an old bug in
fetchSDA()/get_chorizon_from_SDA() related to
LEFT OUTER versus RIGHT JOIN to chtexture table
get_SDA_coecoclass() better handling of
NULL ecoclassref; support for filtering on
ecoclasstypename; not_assigned_value now
applies to ecoclassname, ecoclasstypename and
ecoclassref columns in addition to ecoclassid;
Thanks to Andy Paolucci and Jason Nemecek. Also, added additional
columns from legend/mapunit tables (areasymbol,
lkey, muname).fetchNASIS(from="components") now returns mapunit and
legend information (if loaded in local NASIS database); results now
contain mustatus and repdmu which can be used
to remove components from additional mapunits and non-representative
data mapunits; thanks to Nathan Roeus_ss_timeline result to
ASCIISDA_query(): Fixed POST request that could produce
unhandled errors on network failure (now returns try-error
like other error-causing code)SDA_query(): In the event of a syntax error, error
contents are now returned as a message, not a warning.uncode() and code() now work with
data.table and tibble objects.sp and raster which
means soilDB no longer requires the soon-to-retire
rgdal package.sf and
terra, which have been added to Suggests.
sf, SpatRaster) will be returned by
default rather than Spatial*DataFrame or
RasterLayer from sp and
raster.fetchSDA_spatial() and SDA_spatialQuery()
gain as_Spatial argument; when TRUE equivalent
(backward compatible) sp and raster data types
are returned.options(soilDB.return_Spatial=TRUE)mukey.wcs() and ISSR800.wcs() bug fix for
some instances where the target extent was miscalculated, resulting in
slight differences from requested resolution (res argument)
of result.mukey.wcs() gains a new possible data source
db="RSS" which accesses a Web Coverage Service containing
grids from Raster Soil Surveys in the United States.rmHzErrors argument now
defaults to FALSE. Please let us know if allowing horizon
errors results in unexpected behavior so we can repair various
interfaces that rely on no errors being present. See
aqp::checkHzDepthLogic() for more information on the types
of things that used to result in profiles being removed from results and
for guidelines on how to fix.plotplantinventory
table to get_vegplot_species_from_NASIS_db() result; thanks
to Zachary Van Abbema for suggestion and feedbackuncode() are now cached
within an R session which results in faster query processing times for
users with a local NASIS database connection set up.
db argument to uncode() has been
deprecated. Now, code() and uncode() can use
ChoiceName and/or ChoiceLabel in coding and
decoding ChoiceValue for all data sources.get_SDA_cosurfmorph(): a new function in “SSURGO
On Demand” style. Users can customize the WHERE clause, target tables
and the grouping variable used to calculate proportions (default
by="compname")
get_SDA_property() with
method="weighted average" and
include_minors=TRUE/miscellaneous_areas=TRUE,
thanks to Andy Paolucci and Dylan Beaudette for testing and providing
feedback on the queries.get_SDA_*() methods (except
get_SDA_metrics()) now support input of custom
WHERE clause in lieu of
mukeys/areasymbols arguments and gain a
dsn argument for specifying a local SQLite database or
DBIConnection.downloadSSURGO() for downloading/extraction of
the SSURGO data by survey area from Web Soil Survey.createSSURGO() for building of local databases as
SQLite/Geopackage from one or more SSURGO exports.
downloadSSURGO(), from
NASIS or downloaded from other sources such as https://datagateway.nrcs.usda.gov/GDGHome_DirectDownLoad.aspx.fetchNASIS()
mixColors argument used to toggle color mixing for
pedon horizons with multiple colors per moisture state.
TRUE mixes colors based on reported color
percentage via estimateColorMixture(). FALSE
returns the dominant color in each moisture state, or first record for
cases without colorpct populated.taxreaction and taxfamhahatmatcl to
extended taxonomic informationsoilDB.NASIS.skip_uncode) to bypass
uncode() in NASIS functions (mostly used for debugging or
special database instances that come pre-decoded)fetchNASIS(from="components") where
local database connection was left open was fixedfetchLDM() function to access Kellogg Soil Survey
Lab Data Mart via Soil Data Access or local SQLite snapshot
(https://github.com/ncss-tech/soilDB/pull/243)get_SDA_metrics() for accessing usage metrics for
Soil Data Access, Web Soil Survey and the Lab Data Mart. Thanks to Jason
Nemecek for submitting the base SQL queryget_SDA_interpretation(method = "dominant condition")
now includes total_comppct_* columns reflecting the summed
component percentage of the dominant conditionpedlabsampnum (KSSL Pedon Sample ID)waterYearDay() bug fix for more specific (sub-daily)
date-time formatsfetchSCAN() better handling of empty results in
multi-site queriesfetchNASIS("pedons") no longer includes
pmweatheringfetchNASIS("components") no longer includes
ecositeoriginstringsAsFactors argument to soilDB functions has been
deprecated in favor of a package option
NASISDomainsAsFactor(TRUE) to toggle package option
soilDB.NASIS.DomainsAsFactorfetchSDA_spatial now supports by.col
"areaname", "mlraoffice", and
"mouagencyresp"; thanks to suggestion by Jay SkovlinfetchNASIS fix for multiple site observation records
with surface fragments; thanks to bug report from Brianna WegnerwaterYearDay() use format and timezone for start date
conversionfetchNASIS fix for
from="components", SS=FALSE and queries where no NASIS
local database is present (e.g. from SQLite source)NASIS_table_column_keys dataset of NASIS
primary and foreign keys by table name to include many more tablesfetchVegdata() no longer errors for child table queries
unless site/site observation/vegetation plot records are missingfetchOSD(extended=TRUE) now returns an element
"NCCPI" containing the National Commodity Crop Production
Index summaries. The values returned are quantiles at the 1, 5, 25, 50,
75, 95, and 99% levels over all SSURGO components with
compname matching series for irrigated and
non-irrigated condition.get_SDA_property() now works with mixed component and
horizon-level properties; thanks to Matthieu Stigler for the bug
reportget_SDV_legend_elements() for fetching and
parsing XML for Soil Data Viewer / Web Soil Survey-style symbol themes
for soil interpretations from Soil Data AccessfetchNASIS() pedon and component geomorphic summaries
now include columns for landscape, microfeature, microrelief, 2D/3D
morphometry, and slope shapefetchNASIS('pedons') now uses
simplifyFragmentData() for surface fragmentsfetchSDA() handle NULL component-level results with an
informative errorfetchSDA() now (again) returns mapunit/legend-level
information via get_mapunit_from_NASIS()createStaticNASIS() and queries to SQLite NASIS
snapshots now preserve date/time using RSQLite 2.2.4+ via
extended_types=TRUE argument to
dbConnect()get_soilseries_from_NASIS() now returns
soiltaxclasslastupdated as a date/time and stores the year
in calculated column soiltaxclassyearlastupdatedget_NASIS_table_name_by_purpose()fetchOSD() gains surface shape proportions in extended
shape_across and shape_down tablesSDA_spatialQuery() what='mukey' now works
with db='STATSGO'loafercreek, gopheridge
and mineralKingwaterYearDay() pass through format
argument for POSIX time conversionwaterDayYear() and .formatDates() allow
optional format and tz argument; used for
consistent POSIX time conversion in tests where date/time has
granularity finer than one dayfetchSDA() extensions for better handling of components
with no horizon dataSDA_spatialQuery() and processSDA_WKT()
fully use {sf}, replacing {sp} in these contextsSDA_spatialQuery() gains argument
byFeature to use multiple single-feature queries and
combine the results with a unique feature ID specified by
idcol argument. This allows for specific feature
intersection results without secondary spatial overlay of the polygons
(https://github.com/ncss-tech/soilDB/issues/222)dbConnectNASIS() no longer requires that the NASIS
credentials option be set if the dsn argument is
specified.loafercreek, gopheridge and
mineralKing from latest fetchNASIS()
results.processSDA_WKT(): replaced {rgeos} with {wk} and
{sf}SDA_spatialQuery(): added query_string
argumentget_SDA_property(): fixes for MIN/MAX aggregation
(https://github.com/ncss-tech/soilDB/issues/219)get_component_from_SDA(): fragment size thresholds now
conform with new (clarified) NSSH definitionsfetchNASIS() upgrades: use data.table and base
internally for data aggregationfetchNASIS("components") now “sieves” rock fragments
and human artifacts using same routines as for pedons; respects
nullFragsAreZero argumentfetchKSSL(..., returnGeochemicalData = TRUE) safely
returns 0-length data.frame when no data availableget_OSD(): Handle “NA” and type conversion for JSON
results an convert spaces to underscores for file names as neededfetchSCAN(): graceful handling of timeout and converted
from {plyr} to {data.table};
(https://github.com/ncss-tech/soilDB/issues/161,
https://github.com/ncss-tech/soilDB/issues/184)get_EDIT_ecoclass_by_geoUnit(): graceful handling of
timeoutget_SDA_muaggatt(): add query_string
argument for parity with other “SSURGO on demand” /
get_SDA_*() functionsget_SDA_pmgroupname() and get_SDA_hydric()
now support alternate aggregation methods.
get_SDA_pmgroupname() is
"dominant component", now also supports
"dominant condition" and "none".get_SDA_hydric() is a map unit level
aggregation of components and assigns new classes ; now supports
"dominant component", "dominant condition" and
"none"fetchNASIS(from="pedons") now supports
fill=TRUE argument just like from="components"
to include pedons that have no horizon recordscreateStaticNASIS(): column order should match NASIS,
even if data types require reorder for ODBC driverfetchSoilGrids() bug fixes, updates to metadata and
references in documentation
(https://github.com/ncss-tech/soilDB/issues/201)
nitrogen values (conversion from cg/kg to g/kg)bdod and nitrogen (SoilGrids uncertainty layer
always uses factor of 10 to create integers)progress and verbose arguments for
text progress bar and additional message outputget_SDA_coecoclass() SOD-style method for
mapunit/component level summaries of ecological site and other
vegetation class informationSDA_query() and all functions that call
SDA_query() get proper column class handling (related to
#190), however:
type.convert() may have changed
(e.g. component.wei)SDA_spatialQuery() can now be used to return soil
survey area symbols or geometry using what="areasymbol" or
what="sapolygon", respectivelyget_soilseries_from_NASIS(); now including taxonomic
mineralogy class which may contain multiple parts for series with
strongly contrasting control sectionsget_SDA_*() methods
get_SDA_property(property = ...) and
get_SDA_interpretation(rulename = ...) vectorization over
property/rulename to work with any aggregation method.
query_string argument (default:
FALSE). Set as TRUE to skip submitting query
to SDA, instead returning a string of the query that would have been
sent instead of data.frame resultget_SDA_property: better handling of NULL,
miscellaneous areas, and property-specific weighting
ISNULL(x, 0) logic that affected weighted
averages in presence of missing datainclude_minors=FALSE includes only
components where majcompflag = 'Yes' in resultmiscellaneous_areas=FALSE removes
miscellaneous land types compkind values from resultget_SDA_interpretation: added argument not_rated_value
with default value of NA to set “not rated”” values across
methods/queries. For backwards compatibility with original SQL use
not_rated_value = 99.0phlabresults wasn’t working as expected
and for highlighting some more improvements
(https://github.com/ncss-tech/soilDB/issues/192)get_OSD() TXT and HTML formats now supported (in
addition to JSON) through a common function interfaceget_NASIS_table_key_by_name()
get_NASIS_fkey_by_name(),
get_NASIS_pkeyref_by_name(),
get_NASIS_pkey_by_name(),
get_NASIS_table_name_by_purpose() methods for helping get
information on primary/foreign keys and thematic groups of NASIS tables
(useful for creating SQLite/external snapshots of NASIS tables)get_mapunit_from_NASIS(),
get_legend_from_NASIS() and
get_lmuaoverlap_from_NASIS() now works for “MLRA Survey
Area” areatypename and no longer is limited by constraints
on legendsuituse or mustatusformativeElement argument to
taxaExtent() (SoilWeb taxon extent function)
taxaExtent("abruptic", level = 'subgroup', formativeElement = TRUE)
will get an 800m grid (for SSURGO data in CONUS) showing extent of taxa
that have “abruptic” in subgroup-level taxon namefetchNASIS(from="pedons") result now contains the
"taxclname" (full family-level taxon name) field from the
NASIS pedon table; this value is calculated based on
contents of petaxhistory child tableget_SDA_interpretation and
get_SDA_property now support aggregation
method="NONE" allowing for returning properties, classes
and ratings for individual components or horizons
(https://github.com/ncss-tech/soilDB/pull/181)ISSR800.wcs and mukey.wcs now return a
result that inherits from try-error (and a message) if the
Web Coverage Service query failsDBI and odbc, replacing
RODBCdbConnectNASIS and
dbQueryNASIS for NASIS access with read-only credentials,
fetching query results, and closing the DBIConnection upon
completiondsn argument to specify a local
“static” SQLite file containing NASIS tables, or custom
DBIConnection to a database with NASIS schema
dsn = NULL uses "nasis_local" ODBC
connection to local NASIS SQL Serveraqp::checkHzDepthLogic(), powered by {data.table}get_SDA_property, get_SDA_interpretation,
get_SDA_muaggatt, get_SDA_hydric,
get_SDA_pmgroupnameOSDquery gets a new argument (everything)
for searching the entire documentfetchNASIS(..., rmHzErrors=TRUE) – spurious removals of
data due to missing “extended” records. fetchNASIS now uses
aqp::horizons<- after building a minimal
SoilProfileCollection from NASIS site and horizon tables.
This allows aqp integrity methods to trigger where
needed–preventing unintentional re-ordering or removals of “valid”
horizon data.HenryTimeLine moved to {sharpshootR} packagemukey.wcs() and
ISSR800.wcs() for hitting web coverage service (WCS) for
gSSURGO, gNATSGO, and ISSR-800 gridsROSETTA() for accessing the new ROSETTA
model API (c/o Todd Skaggs, USDA-ARS)fetchOSD(..., extended=TRUE) gains geographically
associated soils, parsed from OSD (thanks AGB)fetchSDA_spatial now can return soil survey area
polygons using geom.src = "sapolygon" with x
as a vector of area symbols (areasymbol) or legend keys
(lkey). For sapolygon results, the
method and add.field arguments work the same
as for mupolygon, only now both geometries can be returned
with fields from the legend table.fetchSDA_spatial now can return STATSGO
gsmmupolygon geometry with db = "STATSGO";
these data are linked to mapunit and (national)
legend just like mupolygon.fetchNASIS / soilDB:::.rockFragmentSieve
now uses fragment RV, soilDB:::.sieve uses
< operator
(https://github.com/ncss-tech/soilDB/issues/1)fetchKSSL(..., returnMorphologicData=TRUE) now returns
redoximorphic features by horizontaxaExtent returns 800m gridded taxonomic
grids for CONUSseriesExtent can now return 800m gridded series extents
for CONUSSDA_spatialQuery can now retrieve SSURGO and STATSGO
geometry c/o dschlaep
(https://github.com/ncss-tech/soilDB/issues/141)data.table; beginning to implement
data.table-aware methods in aqp and
soilDBfetchGDB for querying tabular data from
SSURGO/gNATSGO/STATSGO File Geodatabasesget_NOAA_GHCND() and
get_NOAA_stations_nearXY() for batch queries of NOAA Daily
Climate Data (requires free API token)fetchSDA_spatial with
chunk.size > 1 resulting in duplicate data in
resultfetchSDA_spatial to handle
queries that exceed JSON Serialization LimitfetchSoilGrids for point data queries to SoilGrids
v2 API with SoilProfileCollection outputfetchKSSL(..., returnGeochemicalData=TRUE) now returns
geochemical, optical and XRD/thermal datafetchKSSL related to vectorization, all
arguments vectorized except for bboxKSSL_VG_model output cleaned-up, now returns phi ->
theta functionfetchKSSL is now fully vectorized and builds on new
SoilWeb JSON APIget_concentrations_from_NASIS_db() and
get_phfmp_from_NASIS_db() for fetching Pedon Horizon
Concentrations and Field Measured Properties from NASIS local
databasefetchNASIS(from='components', fill=TRUE, rmHzErrors=TRUE)
in context of new ::hzDepthTests() and non-unique
chiid due to NA values introduced by
fillsimplifyColorData and mix_and_clean_colors
always use CIELAB colorspace for weighted averages, and best-available
metric for transformation to Munsell notationfetchSDA_spatial - new fetch function that simplifies
getting spatial data associated with a vector of mukey or
nationalmusym. The function has options for customizing
result attribute table and is designed to automatically use
makeChunks() to prevent timeout on large queries.aqp::test_hz_logic is now deprecated – refactored
affected fetch functionsfetchNASIS_pedons()fetchNASIS() has @restrictions set automatically if data
are populatedget_RMF_from_NASIS_db()SDA_query() related to multi-line records
(https://github.com/ncss-tech/soilDB/issues/28)sharpshootR added to SUGGESTSfetchHenry() and fetchSCAN() now include water year/day
(Oct 1 – Sep 30)HenryTimeLine() convenience function addedfetchOSD(..., extended=TRUE) when no climate
data availableSDA_query()simplifyFragmentData() and related functions now 4-5x
fasterfetchOSD() now returns metadata when extended=TRUEstringrfetchNASIS() related to conversion of NULL
fragment volume to 0fetchKSSL() can now automatically simplify colors with
simplifyColors = TRUEsiblings()fetchOSD(..., extended=TRUE) gets competing soil
seriesfetchOSD() gets an overhaul, new API and featuresOSDquery()loafercreek and gopheridge sample
datasets and manual page to reflect latest fetchNASISRCurl functions in favor of
httr alternatives (done)reshape to reshape2
(pending)reshape package, will
transition to reshape2 with soilDB 2.0loafercreek and gopheridge
sample datasetssuggests, and checking for package availability at function
runtime. this will make soilDB more portable, as not every user will
want or need all functionality.