Planet Hunters Talk

Lomb-Scargle Periodograms (credit and benchmarks)

  • ajamyajax by ajamyajax

    Awesome fun with L-S periodograms recently! I finally got back to these routines and got them working with Kepler data. Woo-hoo!! I am pleased with how clean the plots can be. Note some of those shown here are just a few more obvious K2 transits I used for testing purposes. And the predicted periods aren't always exact, just "close" at times for some curves.. More detrending will probably help with this, but later for adding that. So still a work in progress as usual. And yes, back to helping with PC finds here soon... I just needed a little programming exercise.

    Credit and Reference:

    "Fast algorithm for spectral analysis of unevenly sampled data"

    http://www.astropython.org/snippet/2010/9/Fast-Lomb-Scargle-algorithm

    "The Lomb-Scargle method performs spectral analysis on unevenly sampled data and is known to be a powerful way to find, and test the significance of, weak periodic signals. The method has previously been thought to be 'slow', requiring of order 10(2)N(2) operations to analyze N data points. We show that Fast Fourier Transforms (FFTs) can be used in a novel way to make the computation of order 10(2)N log N. Despite its use of the FFT, the algorithm is in no way equivalent to conventional FFT periodogram analysis."

    scipy.signal.lombscargle: computes the Lomb-Scargle periodogram:

    http://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.lombscargle.html

    Lomb periodogram of real data, by George B. Moody

    http://www.physionet.org/physiotools/wfdb/psd/lomb.c

    ...

    Development tools:

    Python:

    https://www.python.org/

    GCC, the GNU Compiler Collection:

    http://gcc.gnu.org/

    ...

    EPIC 202137637:

    EPIC 202137637

    EPIC 202126884:

    EPIC 202126884

    EPIC 202072704:

    EPIC 202072704

    EPIC 202066537:

    EPIC 202066537

    EPIC 202065879:

    EPIC 202065879

    EPIC 202126888:

    EPIC 202126888

    HIP 116454:

    HIP 116454

    Note: this period s/b around 9.12, but I only get rounded values returned from the L-S routines I use when plugging in the paper's RV data. Just something more to work on.

    K2 corrected light curve data credit: Vanderburg & Johnson (2014)

    Posted

  • troyw by troyw

    Nice job Aja. I've become pretty familiar with how to implement a fourier series but haven't figured out a fourier transform yet.

    Posted

  • robert_gagliano by robert_gagliano

    No doubt this is a dumb comment since I don't understand programming but why can't you just use this service.
    http://exoplanetarchive.ipac.caltech.edu/cgi-bin/Periodogram/nph-simpleupload

    Posted

  • ajamyajax by ajamyajax in response to robert gagliano's comment.

    Not a dumb comment, that works for others I am sure. The best answer I can think of however, is programmers such as Troy and myself enjoy trying to solve these problems on our own, and customizing the output as well. There is a "yes, I've got it!" factor here you see. (That is the goal anyway.)

    Posted

  • troyw by troyw

    Quite right. It's about the experience and knowledge gained, and the rush of endorphins when a bunch of programming comes together and does exactly what you intended. It's goal-setting and then goal-achieving.

    Posted

  • ajamyajax by ajamyajax in response to troyw's comment.

    Troy, check this out... This is one of my L-S-P's before detrending and then after. What a difference!

    I'm hoping to get more transit candidates with this program that are difficult to see otherwise. But we'll see how it goes.

    Before detrending:

    EPIC 202083828-before

    After detrending:

    EPIC 202083828-after

    Detrended data chart (as-is, just a line chart)

    EPIC 202083828-detrended

    Posted

  • troyw by troyw

    Wow, that's pretty awesome. Have you thought about running known candidates with a wide range of transit depths to see where your sensitivety range is?

    How would this deal with a candidate with TTVs that exceed it's transit duration?

    Posted

  • Artman40 by Artman40

    Does this mean at least one more planet may be detected from engineering data set?

    Posted

  • ajamyajax by ajamyajax in response to troyw's comment.

    I’m still working on finding a balance between sensitivity for small planet detection and strongest signal detection. If the period is known or looks promising anyway, I can mask the transit before detrending as you mentioned on PH elsewhere (and then replace them after for best signal detection). So currently I have to look at a “before flattening” chart for faint unknown signals because detrending just wipes those clean. And there might not be a workaround for that. But I also need to run more known candidates as you suggest to see what they look like here, including those with known TTV (in theory, will just see a wider periodic signal for most of them). So as usual, these kind of projects are a work in progress for a while; for me anyway.

    Posted

  • ajamyajax by ajamyajax in response to artman40's comment.

    Yes, that’s the beauty of these charts when I/we can get them to work right: all periodic events should rise above the noise and be visible to some degree. The very short period spikes you often see on the left side of these charts are mostly stellar frequencies as I understand it. We once looked at a nice published study on PH1 where the star frequencies were identified and masked out, leaving only the possible periodic signal(s) of a disintegrating short period planet. (Isn’t science fascinating?)

    Posted

  • arvintan by arvintan

    Mark and Troy, you guys are among my programming idols. These kinds of discussions with the plots are the reason why I learned Python in the first place. 😃

    Any hint where I can start learning what you guys do? I'm referring to the numerical analyses.

    Posted

  • ajamyajax by ajamyajax in response to arvintan's comment.

    Thanks much Arvin, but all I can suggest is to experiment with the example code you can find here and on the internet. That's how I learned to do everything with Python. You'll figure it out also! Best of luck. Mark

    Posted

  • troyw by troyw in response to arvintan's comment.

    I personally never learned python. I have been using PHP and javascript to keep everything web based. The first step is getting data into your script. I do this by copying a txt file into a text field (for AKO) and programatically split the data into an array for 'time' and an array for 'values'.

    Posted

  • arvintan by arvintan

    Thanks, Mark and Troy. Will definitely try some coding on these things.

    Posted

  • Martti_Holst_Kristiansen by Martti_Holst_Kristiansen

    Hi Arvintan,

    What operating system do you have? I am using Windows and I have really tried to get python to work. Still without luck.

    Btw, looks good Mark!

    Posted

  • ajamyajax by ajamyajax in response to Martti Holst Kristiansen's comment.

    Thanks Martti, as always we'll see what else we can find that looks interesting in the data.

    About Windows and python, try the executables here for your library installs:

    http://www.lfd.uci.edu/~gohlke/pythonlibs/

    Usually they work just fine. But python itself should be running ok on your PC first. If you can go to the command prompt, type python, and get the interactive session you are probably ok. If not make sure your Windows environment has a path set to the python subdirectory. That's a good place to start anyway.

    Hope this helps. p.s. type exit() to return from the interactive session to the DOS prompt.

    Edit/Update:

    I noticed the site I posted above has changed the LIB files they posted from executable format to WHEEL compressed format files (I believe they are called). These are new to me also, but seems like Windows users can install those updates with the PIP utility in the \pythonxx\scripts folder. Tried a few and seems to work.

    Also there are new python 2to3 upgrade tools available. Tried those as well and they mostly worked, but be sure to back up your programs before modifying just in case.

    http://python-future.org/automatic_conversion.html

    Posted

  • ajamyajax by ajamyajax in response to ajamyajax's comment.

    Forgot to mention yesterday that PyKE and Kian's programs I think, are only python 2.7.x compatible... You can have both python 2 and 3 on your PC, but you need separate BAT files to run what you want or upgrade them yourself. To keep it simple you might want to only install python 2.7.x. There is a conversion utility that assists in converting source from 2 to 3 named just that, 2to3.py in the \python3x\tools\scripts subdirectory. But just like in looking for planets, setting up and using python can take a lot of time and patience.

    Posted

  • arvintan by arvintan in response to Martti Holst Kristiansen's comment.

    Martti, I'm currently using Windows 8. But I've also gotten version 2.7 to work on Windows 7. It was fairly straightforward for me to install it and use the IDLE IDE. Were you able to open the IDE?

    Posted

  • arvintan by arvintan in response to ajamyajax's comment.

    Mark, do these graphs require PyKE? I hope not. Just looking at the installation process intimidates me.

    Posted

  • ajamyajax by ajamyajax in response to arvintan's comment.

    Arvin, none of these graphs use PyKE. I just mentioned PyKE because Dolorous Edd and others run Keppixseries charts, which are PyKE and python.

    Posted

  • Martti_Holst_Kristiansen by Martti_Holst_Kristiansen

    Sorry, I actually think I got pyhton 2.7.x working but I never got PyKE to work, and I do not even remember all the things I have tried. It was a couple of months ago and a bit confusing. Maybe I should give it another go next weekend, and if I do, I will add a guide and maybe we can find of where the problems are.

    Posted

  • ajamyajax by ajamyajax

    All right, making some progress with K1 L-S-P benchmarks and detrending tests.. This is KIC 10666592/KOI 2.01/Kepler-2 b/HAT-P-7 b from a sample of data between 818 and 870 BJD. It appears a data subset can be as useful as the entire data set and is easier to see and test. The small red-ish transit I highlighted on the standard data plot is the albedo effect secondary, where the luminous planet presumably gets eclipsed by its star. And detrended data produced the clearer chart as expected, but also removes weaker signals in general.

    10666592,KOI 2.01,CONFIRMED,2.204735365,121.3585723,3.88216

    PB1
    LS1
    LS2

    Posted

  • ajamyajax by ajamyajax

    And these are KIC 7303287/KOI 1353/Kepler-289 transit signals in a detrended data L-S-P, with the strong primary P=125.x signal removed so the weaker signals are more visible.

    I got stronger TTV signals there around these BJD values (34.54, 69.09, 103.63, 138.17, 172.74, 241.81) but only those labeled are possibly planetary, and the rest are harmonics. I proved that I believe, because removing the four transits I processed (three confirmed TTV planets, plus my new 1.91 period ratio speculative transit at 240.x) caused the harmonic signals to disappear. Also, I noticed my L-S-P signals lean toward the high end of the TTV ranges or just above that, so perhaps that is how the L-S-P subroutines see them. I'll look into that more in later tests.

    LS1

    (And here are some plots to supports my new PC at P=240.x theory, but only two possible TTV transits are "more visible" so this remains speculative.)

    OC4
    T4A
    T4B

    Posted