Bob is going to talk first, since he has to leave a bit early today. He said that everything is going great with the MSQL Servers… and that is all.
Josh reported that now all 6 of the RFI detection algorithms are in place. One of them is working pretty slow: it takes about 4 hours to go through the data. Eric suggested that if we need it to go faster, we can just have to redo the whole thing in C++.
In comparing the algorithms, Josh noticed that the polarized one works the best—but it is not necessarily true that a signal from ET would be polarized, there is no way to tell. So, this algorithm may not be the best method for our purposes. Josh will continue to work on evaluating and developing all of the algorithms, and making sure that they don’t throw away signals that we may be interested in.
Adam has been working on trying to detect microsecond dispersed pulses. At first, he had a little bit of trouble getting some of the information out of the files, but he figured it out so he is going to start doing some waterfall plots. Andrew suggested that he take a look at Phil’s processes to see what normalization he used, so that when he produces a plot it is not dominated by RFI.
We’re using Phil’s code. Adam wondered there any other sort of programs or formats that we use to look at that sort of data? It was settled that he’d chat with Eric about it, because he’s an IDL person.
We had a little problem with the data for SERENDIP V. But we have it ironed out now, and we have a script that checks everything every 20 min to make sure everything is working fine. If it isn’t it restarts everything.
Eric now has the table defined for the Zone RFI detection. It’s going be a small table; so he’s not going to bother with fragmenting it. It will have maybe one entry a day. But that could mean a lot of entries. Tomorrow Bob will look and see how much space we have to add to Astropulse.
Eric submitted the three papers to the Bioastronomy Conference, and they were all received. One will be credited to Kevin, one to Josh, and one to Eric.
Matt recently did some cleanup work on the code for the Radar Blanking project. He got it working this morning, it is a slow and steady progress, but hopefully it will work out to be a really valuable tool.
In NITPCkr News: Matt hasn’t been running it everyday because it seemed to be blocking the science database. We might want to discuss what we want to do next with this project. We originally launched NITPCkr without a lot of fanfare. But the purpose of that is that we want to wait until we get the zone RFI under control, and we don’t want to implement tools for a lot of user feedback until the data is meaningful. For now, matt will just keep generating candidates by hand every couple of days or so.
Dave has been working on a talk he’s going to give at the upcoming NVIDIA conference on GPU computing, taking place October 2-4.
The guys behind Einstein at Home will be here for a few days after the conference. They have a CUDA version and an Open CL version of their application. Open CL code may be something we could consider for SETI in the future. Although, the new versions may end up being very similar to our existing kuda code. We might want to meet with the Einstein guys and chat with them about this while they’re here.
Dan brought up an important question: Does Open CL have the same good scientific sub-routine that kuda does, such as FFTs? Well, the answer is: not right now. But in the future it will be developed because a lot of people are switching over to Open CL, including NVIDIA.
Jeff has two weeks before he leaves on his trek through Nepal, so he is working on trying to get stuff done before that.
One thing he noted is that we have about 1600 tapes with data on them, or about 70 terabytes of data. How would we transfer this data to a better storage format? Tapes tend to last longer than you can find something that can read them—and it would be a good idea to get them over into some form of new media.
Jeff also reported a problem: every few days our data recorder crashes. It has been doing this since about the time we updated it (a couple months), and it happens in the process of closing one file and opening another one. We had lots of de-bugging information store, and after the restart these files were just not there. The radar system logs are there, but the data system logs are not. Eric recommended that could be because logs are being rotated. We might want to look at the log rotation scripts and make sure that they are not being deleted. Also, check if the file names end in .log!
Jeff has continued to work on the RFI framework in conjunction with NITPkcr, and hopefully by the time he goes on his trip he’ll have something that he can hand off to Eric and Matt.
Also, we’ve been having some splitter beam problems that are somewhat mysterious. Occasionally the splitter will produce duplicate work unit groups. This seems to be really only a tiny problem—it causes us to produce 5% more work. But the worst case is that it is indicative of some larger problem in the data. We’ve seen this happening over the last couple of months, and usually it happens with only the first beam in the splitter. And we’ve only been able to discover it happening after the fact. Further investigation is required.
Dan announced that the Astronomy Department at Berkeley got some money to buy a cluster. It’ll probably end up being a very conventional setup, but we can use it if we need it.
Also, Andrew and Dan have been working with the Los Alamos group. The statisticians out there have done some very clever things with the data, and developed some useful algorithms that have shown better false alarm rates. They have applied all these algorithms to two hours of data. Now they have to work on applying it to 500 hours of data, and make let them go real-time.
Dan and Los Alamos have chatted with IBM about their new technology, Infoshpere. The people at Los Alamos are going to maybe move their algorithms in to InfoShpere and this whole streaming architecture. They are interested in transients, and they are interested in working with us, so is IBM. Exciting news!
Andrew and Dan are heading off to South Africa in the later part of next week, so they’ll be here for the meeting but will be gone for the next couple of weeks.
That is all for this week! Thanks for reading!