Hi.
we have a CR5000 which has a 10Hz measuring frequency (Eddy covariance, RH sensor and net radiometer), and the data is stored on the logger's CPU. The logger is connected to a computer that, using the task manager, collects data every 5 minutes, in order not to lose data.
Since the measuring frequency is so high the output file i am getting us huge, for one, and second thing is that we like to look at 24 hour periods. My question is - how do i program the logger to create a new output file every 24 hours?
I have tried using "TableFile" but it doesn’t allow me to apply the CPU as device, it accepts only CRD, but i am not using a card.
Another option is using the Split option in the LoggerNet. But i much rather have the program doing it automatically, if at all possible.
One last thing - i need it to be CSV array, but that i manage through the data table options in the setup.
Ideas?
Thanks,
Mariela.
One easy way to break up your existing file is to use FileFormatConvert.
You can find that utility in the installation directory. On 64 bit Windows, it is in the folder:
C:\Program Files (x86)\Campbellsci\FileFormatConvert
Split gives you more flexibility.
A simple solution some customers do is rename the .dat file. Loggernet will then create a new file at next data collection. You can use Task Master in Loggernet to run a command line script after midnight each day.
JDavis has given a great summary of what you can do on the PC side to solve this issue.
If you are determined to have the datalogger/CRBasic program create the 24-hour file, then the TableFile instruction seems to be what you need. Even though you are hesitant to use a compact flash card (CRD), you should still consider it. Most CR5000 users who collect data at a high rate (like you are doing) and deal with the associated large data files use a compatct flash card (CRD device):
https://www.campbellsci.com/cfmc2g
On the CR5000 you will need a CF1 for the card to work:
https://www.campbellsci.com/order/cf1
The use of a CRD storage device reduces the risk of lost data pretty significantly.
The TableFile instruction does not allow the CPU drive as a destination for its otput file because the CPU drive resides in serial flash memory and typical TableFile usage would exceed the CPU's expected life (i.e. wear it out) in a very short time due to the large amount of read and write operations involved. The CPU drive probably isn't big enough to hold your entire 24-hour file anyway.
The TableFile output device options are : CRD, USB and USR. The USR drive is not available on the CR5000, so your options are to use a compact flash card (CRD) or purchase an SC115 device (USB) which connects to the CS I/O port.
https://www.campbellsci.com/sc115
Options 12 to 15 in the TableFile instruction give you a TOA5 file without a header, which is pretty close to CSV output.
Do you have OS v7 (Std7) loaded on your CR5000 ? There were some TableFile updates made to the operating system between version 6 and version 7. Version 7 supports TableFile with "option 64" which is discussed here:
https://s.campbellsci.com/documents/us/technical-papers/write-high-frequency-data-to-cf-cards.pdf
CR5000 OS :
Another option, which is less precise but might be OK, is that LoggerNet has a setting that will let you set the maximum file size for a data file. By default, this is set to -1 (so no maximum file size). However, you could roughly estimate what 24 hours worth of data would be and then set the maximum file size to that. Of course, the range of data contained in the file would be "rough" as well, but if your biggest concern is the size of the files you're dealing with, it might suit your needs. This option is found in the Setup window, Tools menu, LoggerNet Server settings.
If you were collecting data from the Setup window, and less frequently than 5 minutes (say, every 8 or 24 hours), another option would be choosing the File Output Option of "Create New File" (on each data collection). I do understand, however, that you don't want a bunch of 5 minute files!
Best, Dana
I should add that if Split will work for your needs, you can set it up as a task to run automatically in the Task Master. The Task Master has a nice example of this in its help file.
Dana
Hi all.
Thanks for the replies.
Well, as I tried to explain, our goal is to have the data accessible online all the time. We used to have a CRD, because the files are so big. we used to go to the field, 2-3 times a week, change the card, go to the office. But since we de decided to connect the logger to a field comp, we had to stop using the CRD, since we can’t access its data from LoggerNet, right? That is when I started downloading the data every 5 minutes using the schedule task.
So I can’t use the TableFile option.
The FileFormatConvert is good for a close file, or can use it on a read only file?
Can i use an if/end command? Program something like, if time isn’t midnight write to this output file but if it is midnight open a new file. Or rename the file, and then it will automatically open it?
I don’t have much experience in programming; this is why I am asking.
I will play around with the options you suggested.
Thanks,
Mariela.