|
|
| (7 intermediate revisions by one user not shown) |
| Line 1: |
Line 1: |
| − | ==CRSP, COMPUSTAT, CCM ==
| + | Chicago Booth is going to retire the desktop tool [[CRSPSift]] by end of 2024 and all the other tools based on fileformat 1.0. |
| − | === CUPL Tools ===
| + | |
| − | ==== access to your remote text terminal ====
| + | |
| − | ===== require credentials =====
| + | |
| − | email the VDC sysadmin to provide credentials and get your remote text terminal prepared.
| + | |
| − | <ul>
| + | |
| − | <li>
| + | |
| − | host: crsp.vdc.univie.ac.at
| + | |
| − | </li>
| + | |
| − | <li>
| + | |
| − | user: your username
| + | |
| − | </li>
| + | |
| − | <li>
| + | |
| − | password: your password
| + | |
| − | </li>
| + | |
| − | </ul>
| + | |
| | | | |
| − | ==== VPN ====
| + | Fileformat 2.0 has been recently released and (re)programming on our side has already started accordingly. |
| − | If working from home you have to establish a Virtual Private Network (VPN) to Uni first ([https://zid.univie.ac.at/en/vpn/ see ZID/VPN]).
| + | |
| − | | + | |
| − | ==== SSH ====
| + | |
| − | If working from Windows, you might need to install an SSH-client ([https://www.putty.org/ like PUTTY]) to use Secure Shell (SSH) to connect to your terminal. Accept the server key.
| + | |
| − | [[File:3_accept_key.png|left|frameless|200px|caption]]
| + | |
| − | Also see [https://zid.univie.ac.at/online-speicherplatz/anleitungen/sftp/ ZID's explanation]
| + | |
| − | <br clear=all>
| + | |
| − | | + | |
| − | ==== usage of a terminal ====
| + | |
| − | Now you work remotely and use the remote host's command line (which is the bash shell, don't type the beginning "$", which is the prompt):
| + | |
| − | [[File:4_terminal_session.gif|left|]]
| + | |
| − | <br clear=all>
| + | |
| − | the midnight-commander (mc) provides a built-in editor mcedit (exit it with Alt+0)
| + | |
| − | <pre>$ mc </pre>
| + | |
| − | <pre>$ mcedit ts_samp7.rqt </pre>
| + | |
| − | e.g. use '''''ts_print''''' to process a requirementfile '''''ts_samp7.rqt''''' and get the output '''''ts_samp7.out''''':
| + | |
| − | <pre>$ ts_print ts_samp7.rqt</pre>
| + | |
| − | <!--
| + | |
| − | [[File:7_ts_print.png|left|frameless|500px|caption]]
| + | |
| − | <br clear=all>
| + | |
| − | -->
| + | |
| − |
| + | |
| − | use mc / mcedit to look into ts_samp7.out:
| + | |
| − | <pre>$ mcedit ts_samp7.out </pre>
| + | |
| − | [[File:8_ts_samp6_out.png||frameless|400px|caption]]
| + | |
| − | <br clear=all>
| + | |
| − | <br/>
| + | |
| − | use e.g. this command to get the switches for the command '''''crsp_print''''':
| + | |
| − | <pre>$ crsp_print /d1 /var/crsp/2019-07/crspdata/cmz201807</pre>
| + | |
| − | | + | |
| − | ==== download the results onto your desktop / laptop ====
| + | |
| − | use an SFTP client like [https://cyberduck.io/download/ CyberDuck] on a Mac or [https://zid.univie.ac.at/online-speicherplatz/anleitungen/sftp/ WinScp] on Windows to download the results (e.g. ts_samp7.out) onto your desktop for further processing. Provide your SSH credentials:
| + | |
| − | | + | |
| − | [[File:10_cyberduck_credentials.png|400px]]
| + | |
| − | <br clear=all>
| + | |
| − | | + | |
| − | ==== documentation====
| + | |
| − | Please read some manuals to gain experience with the ts_sampleX.rqt files in your home directory
| + | |
| − | <ul>
| + | |
| − | <li>
| + | |
| − | [http://www.crsp.org/products/documentation/cupl-386-guide CRSP Utilities and Program Libraries (CUPL)]
| + | |
| − | </li>
| + | |
| − | <li>
| + | |
| − | [https://w3.loibl.com/uni/ccm_data_guide_0.pdf CCM Data Guide]
| + | |
| − | </li>
| + | |
| − | <li>
| + | |
| − | and maybe the [https://w3.loibl.com/uni/xf_understanding_the_data.pdf xf-understanding_the_data guide].
| + | |
| − | </li>
| + | |
| − | <li>
| + | |
| − | collected [https://w3.loibl.com/wiki/index.php/CRSP#Usage thoughts] .
| + | |
| − | </li>
| + | |
| − | </ul>
| + | |
| − | === WEB Access R ===
| + | |
| − | * apply for login-credentials andreas.loibl@univie.ac.at
| + | |
| − | * use sftp to upload / download your date (like with your [https://zid.univie.ac.at/online-speicherplatz/anleitungen/sftp/ Online-Speicherplatz]
| + | |
| − | * http://calc.isor.univie.ac.at:8787
| + | |
| − | * explore some commands, for the sake of functionality only (not to make any sense)
| + | |
| − | ###### Example#1
| + | |
| − | # search for "EGSA"and perform some actions
| + | |
| − | # comment, not interpreted
| + | |
| − | ? system
| + | |
| − | ? system.time()
| + | |
| − | help("crsp.data.usstock") ## we have licensed US-Stock 1925 and 4 US-Indices and CCM
| + | |
| − | system2("uptime") ## load average < 10 ! (otherwise too many concurrent users)
| + | |
| − | library(data.table, crsp)
| + | |
| − | setwd('~/crsp')
| + | |
| − | getwd()
| + | |
| − | myheaders <- data.table(crsp.data.usstock('sfz_hdr'))
| + | |
| − | myEGASHeaders <- myheaders[HTSYMBOL %like% "EGAS" & BEGDT > "1986-01-8"]$KYPERMNO
| + | |
| − | myMonthly <- crsp.data.usstock("sfz_mth")
| + | |
| − | nrow(myMonthly)
| + | |
| − | ncol(myMonthly)
| + | |
| − | head(myMonthly,10)
| + | |
| − | MyEGASData <- myMonthly[myMonthly$KYPERMNO %in% c(myEGASHeaders),]
| + | |
| − | head(MyEGASData)
| + | |
| − | tail(MyEGASData)
| + | |
| − | plot.ts(MyEGASData$MPRC)
| + | |
| − | ######## Example #2
| + | |
| − | # read tickers from one file perform some operations and write results to another file
| + | |
| − | # myTechies <- read.csv('/home/su/crsp/techcompanies.tickers', header = FALSE)
| + | |
| − | myTechies <- scan(file = 'techcompanies.tickers', what = 'character', sep = ',')
| + | |
| − | myTechiesHeaders <- myheaders[myheaders$HTSYMBOL %in% c(myTechies)]
| + | |
| − | #fileDsk <- tempfile("MyTechiesHeaders")
| + | |
| − | fileDsk <- "MyTechiesHeaders"
| + | |
| − | #setwd("~/R_workspace/from_uspace/privat/crsp_sample1/")
| + | |
| − | write.table(t(myTechiesHeaders), file='./MyTechiesHeaders')
| + | |
| − | # file.show(fileDsk)
| + | |
| − | # if(interactive()) file.show(fileDsk)
| + | |
| − | unlink(fileDsk) # tidy up
| + | |
| − | myTechiesData <- myMonthly[myMonthly$KYPERMNO %in% c(myTechiesHeaders$KYPERMNO),]
| + | |
| − | plot.ts(log(myTechiesData$MPRC))
| + | |
| − | | + | |
| − | | + | |
| − | ==== Documentation ====
| + | |
| − | [http://www.crsp.org/products/documentation/overview-4 in the crsp.org universe.]
| + | |
Fileformat 2.0 has been recently released and (re)programming on our side has already started accordingly.
