Skip to content

adding processing scripts for ticket #19667 Russell #12

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
angelchen7 wants to merge 3 commits into
base: master
Choose a base branch
from
Open

adding processing scripts for ticket #19667 Russell #12

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
202cc66
adding processing scripts for ticket #19667 Russell
Mar 2, 2020
451cd13
Merge branch 'master' of https://github.com/NCEAS/data-processing
Aug 5, 2020
a13e5f0
adding another processing script
Aug 5, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions R/Angel Chen/ticket #19667 Russell.R
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
#ticket #19667: https://arcticdata.io/catalog/view/urn:uuid:6a64c7ad-aa08-424a-9cf9-8a5747436813
#formatted like this https://www-air.larc.nasa.gov/missions/etc/IcarttDataFormat.htm
#spending time to do https://learning.nceas.ucsb.edu/2020-02-RRCourse/data-cleaning-and-manipulation.html

#making attributes for Barrow_Spectra_Barrow_20080302_R1_thru20100305.ict
#grab the names of the columns using names() and make that a data frame (using data.frame()
#split the names into wavelength and frame number columns using separate()
#create descriptions of the attributes using a combination of mutate() and paste(), descriptions can match a format like "absorbance at x nm, frame y"
#add units, etc using mutate (unit can be dimensionless

#1878 total variables


pkg <- get_package(adc, 'resource_map_urn:uuid:6b36ffd1-98f3-415d-a1b7-822a67745a1b', file_names = TRUE)
doc <- read_eml(getObject(adc, pkg$metadata))

dat <- read.csv("~/ticket #19667 Russell/Barrow_Spectra_Barrow_20080302_R1_thru20100305.ict", skip = 1906)

names_df <- data.frame(names(dat))
names_vector <- as.vector(names_df[[1]])

descriptions1 <- c("number of seconds elapsed since 03/02/2008 in Coordinated Universal Time (UTC), recorded when a day began", "number of seconds elapsed since 03/02/2008 in Coordinated Universal Time (UTC), recorded when a day ended")

wavenumbers <- seq(400, 4000, by=1.92)

@jeanetteclark jeanetteclark Mar 2, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this works since you know what the step is for the wavenumbers (and it is an equal step across all the data), but the way I would have done this would leverage the values already in the columns. See suggestion below which leverages the dplyr functions

@jeanetteclark jeanetteclark Mar 2, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

w_atts <- data.frame(attributeName = names(dat)) %>% 
  mutate(wavenumbers = gsub("_", ".", attributeName)) %>% 
  mutate(attributeDescription = paste("absorbance at wavenumber", wavenumbers)) %>% 
  select(-wavenumbers)

@jeanetteclark jeanetteclark Mar 2, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can then use that to build the rest of your attributes, and stick on the top two using rbind or add them in shiny_attributes

descriptions2 <- paste("absorbance at wavenumber", wavenumbers)

scales1 <- c("ratio", "ratio")
scales2 <- rep("ratio", 1876)

domains <- rep("numericDomain", 1878)

format_strings <- rep(NA, 1878)

definitions <- rep(NA, 1878)

units1 <- c("second", "second")
units2 <- rep("dimensionless" ,1876)

number_types <- rep("real", 1878)

missing_codes <- rep(NA, 1878)
missing_explanations <- rep(NA, 1878)

attributes <- data.frame(
attributeName = names_vector,
attributeDefinition = c(descriptions1, descriptions2),
measurementScale = c(scales1, scales2),
domain = domains,
formatString = format_strings,
definition = definitions,
unit = c(units1, units2),
numberType = number_types,
missingValueCode = missing_codes,
missingValueCodeExplanation = missing_explanations,

stringsAsFactors = FALSE)