Planet Hunters Talk

Extracting lightcurves with AKO-TPF

  • troyw by troyw

    I've been working on a tool to extract lightcurves from Kepler and K2 Target Pixel Files (TPFs). Here is a link and tutorial for this tool. Please keep in mind when using this tool that it is a constant work in progress and you may experience buggy behavior at times. I'm aware of several flaws that I hope to iron out over the course of K2. I'm sure by the time this tool is perfected, it will have also outlived its usefullness, but in the meantime I hope it can at the very least help people identify PHOI's (Planet Hunters Objects of Interest) ahead of the release of more well extracted light curves by professional teams and algorithms.

    FIrstly, I'd like to thank Andrew Vanderburg for his guidance. Although my method is vastly different, I learned a great deal of what to and what not to focus on. Thank you, Andrew.

    The link for this tool is at

    www.extrasolar.us/AKO/AKO-TPF.php
    There are issues with using this tool in Firefox. I use Chrome, and I haven't tried any other browsers. Use at your own risk.

    This tutorial will show you how to add data to AKO-TPF and extract a light curve. We will use data from the 9 day Engineering Test for the target WASP-28b EPIC 60017806

    The data for this file was taken from the FITS file provided at MAST. DATA FILE

    Extracting data using FV will be shown in a separate post in this tutorial.

    1. Locate and click the "Add Data" button in the upper left corner of the control panel.

    2. My computer is not the fastest and I have never tested this on any other computer. I've found that the automatic spellcheck provided in Chrome slows down the act of copy and paste into the text field, so I would recommend turning it off in the browser settings. Even with this setting turned off, it could take several seconds to over a minute (depending on the size of your data) for the data to appear in the text field. This is a limitiation I have not found a way around.

    Copy and paste the contents of the data file into the text field that appears. Click "Insert".

    Once the "Insert" button has been pressed, the browser will take several more seconds to ingest the data. Be patient. Copy and pasting large amounts of data, such as the 50x50 TPFs provided for Campaign 1 may crash your browser tab as it does mine. Fortunately, you can break up the data and add it sections by again clicking repeating step 1 and 2 until as many times as necessary to insert all your data. (I break up the C0 50x50 TPFs into 3 parts, but the 17X17 TPFs for C1 (longer data set but less pixels) can usually be added in one part).

    1. The text field where the data was added should now dispappear. Scroll to the top of the control panel and click the red "Generate" button.

    2. This file is a 50x50 pixel area on Kepler's CCD. The "Pixels per row" option is set to 50 by default. If your TPF is a different size, you can change the "50" to the number of horizontal pixels in your TPF by click in red "50", using one of the buttons next to the "50", or using one of the presets in the next row down.

    FV shows this dimension on the aperture row when a file is opened.

    enter image description here

    You should also see several fields in yellow fill in. The following list gives a description for each item:

    • Image Minimum and Image Maximum: These values are used to calculate the grey value of each "pixel" in the image AND the graph at the bottom of the screen.

    • Frame : This is the number of frames contained within the TPF file. Each 'frame' is equal to one datapoint in the resulting lightcurve and corresponds to a time which is shown on the graph at the bottom of the page.

    • Minimum time and Maximum time : These numbers can be adjusted to change only the graph at the bottom of the page. (In order to zoom in on a range of data). If you happen to change this, you can reset these fields to their default start values by clicking Minimum time and Maximum time.
    1. The default settings for AKO-TPF after data has been entered is intended to dynamically track the centroid positions of a target that you choose. On the control panel, you'll notice that Track and Follow are turned on. When enabled, these functions work together to build your light curve.

    You'll also see that the "Graph option" is set to "4". This means that the graph on the bottom will show one of two graphs depending on whether or not the centroids for a target have been tracked. Since they have not yet been tracked, the graph will show the flux values for just the pixel that is covered by your mouse pointer. In this case, WASP-28 is in the center of the TPF image.

    enter image description here

    The above image shows some data on the graph at the bottom. The reason why is because the target moves to a different pixel 114 frames into the data set. Moving the mouse around this target will show various graphs. Quickly move your mouse off of the TPF image to keep a pixel near WASP-28 highlighted. (A future update of this software will use keyboard shortcuts or something similar to eliminate this admittedly frustrating exercise. (get the mouse off the image before it moves with the mouse to a far off pixel)).

    At this point, the target is ready to be tracked.

    1. Locate and click the play button enter image description here at the top right of the screen.

    You should notice that the graph at the bottom changes with each frame. This is building a lightcurve based on (only) the pixel movements. It combines the central pixel and an additional 3 pixel halo around the central pixel. This is not the graph we'll be using for this tutorial, so click the button that is labeled Rcurve(6). This is just short for 'reconstructed light curve' and it happens to be option 6 of the available graphs for the bottom of the page.

    Here is what Rcurve is showing: With each frame, the centroid position of your target is calculated. Although the method for calculating the combination of pixels is (in my opinion) somewhat of a hack, it also seems to work very well until I can find a faster replacement calculation.

    An image is drawn (if you scroll to the bottom of the control panel, you should see it appear) that represents an aperture of 7X7 (TPF) pixels. A red circle is drawn at the calculated centroid position of your target for each frame.

    enter image description here enter image description here

    Then, each pixel section in the image is evaulated to see how many red vs non-red pixels occupy a pixel section for a total of 49 pixel sections. So the image above on the right has a higher percentage of the bottom central pixel than the frame on the left, which has zero percent of the circle covering that pixel.

    1. When the last frame has been reached, you can get the data from the newly extracted curve by clicking the
      Reconstructed curve data button. Be sure that your browser is set to open new windows. You can now copy and paste the data into Excel or a similar program to view.

    If you change the "Image minimum" to 29000 and the "Image maximum" to 32000, you should see good representation of the variations in the reconstructed light curve.

    You'll see another image next to the TPF image that shows a series of points across a grid. This is a graphical representation of the centroid postitions of your target. The color of the points represent the quadrant that the points fall in. You can look at an alternate version of this image by checking the "Alternate centroid image" box, which shows the points as black to white from the earliest time to the last time in the data respectively.

    enter image description here

    Tips

    Turn of tracking and click "Sm Aperture(7)" or "Lg Aperture(8)" and run your mouse over various targets in the image to get a 25 and 49 pixel combination respectively, of flux in the bottom graph. This is a good way to "browse" the field and get a sense of each targets lightcurve.

    Clicking "Min/Max(1)" will give you a graph of just the pixel you are covering with your mouse pointer on the image.

    Turn off the "Reconstruct" check box for quicker load times when 'browsing' the TPF image with your mouse, or if you just want to obtain the centroid values of a target.

    While tracking a target, turn off "Pixels" and/or "Graph" at the top right of the control panel in the "Animate while calculating" section.

    Click "Start" or "End" to jump to the start or end frame. These adjustable numbers also set the loop in and out when in use.

    When a target is being tracked, the brightest quadrant within a 9 pixel aperture is also being stored for later use. This is what the different colors indicate when looking at the reconstructed graph (purple=top left quadrant ,Blue=top right ,red=bottom left, green=bottom right.) The colorful adjustments in the middle of the control panel affect the final flux based on the quadrant and sub pixel position of the target and can either add or subtract to each quadrant by adjusting these. This does not always work well, but it has its moments. The arrows increase or decrease these values by the adjustment number set under the Max adjustment (in red, whose default value is "100")

    You can delete the background flux that affects all the pixels (fairly) uniformly by turning on the "Get background" option and clicking an area of the TPF image that contains as little flux as possible (dark pixels). Then, unclick the "Get background" option. When reconstructing a curve, you may have to set the "Background multiplier" to anything from 3 to almost 8 to flatten out the final curve. (This is usually my FIRST step, but it is not necessary in terms of calculating centroids and can be done at any time).

    Clicking the "Centroids" button will bring up an image in a new window showing a larger version of where the subpixel centroid positions fall.

    Clicking the "XYArray" button will give you Time,X centroids,Y centroids, and the pixel number targeted for that time.

    Pixel numbers are based on the width of the TPF (50 pixels in this case) X the number of pixels from the bottom + the number of pixels from the left. The yellow information panel at the bottom of the screen shows the direction of a bright pixel in relation to your mouse position if the "Track" option is checked, and shows the pixel number if "Track" is turned off.

    Click the graph on the bottom at any time to open a window to export the graphed curve.

    You can control the size of the reconstruction aperture (the red circle) by adjusting the "outer" text field. The "5" is the diameter, in TPF pixels of the aperture. I've found a diameter of 5 seems to work best. The "size" textfield next to it is how big to make the image to base the calculation from. The default is a 100x100 pixel image, but can be set to anything. However, my computer does not handle anything over 300 very well, and I've found that resolution of the light curve does not vary much between 200 and 300, but does vary between 100 and 200. For calculation speed, the default is set to 100. I tend to do a low resolution pass, and if I see something interesting, I make the image size larger to increase resoluition.

    There are a few other functions that I will go over as soon as I can, but for now this should give a good start.

    Posted

  • troyw by troyw

    To extract TPF files as txt files to enter into AKO-TPF, you'll need FV software.

    http://heasarc.gsfc.nasa.gov/ftools/fv/fv_download.html

    1. Open a TPF file with FV.

    2. Choose the "Select" option from the TARGETTABLES row.

    enter image description here

    1. Choose "Clear all" and then choose "Time" and "Flux" and then "Display Table"

    enter image description here

    1. Choose File > Export as Text

    enter image description here

    You now have a text file to copy and paste into AKO-TPF

    Posted

  • troyw by troyw

    After a small technical update, and following the tutorial instructions, you should end up with a light curve for WASP 28b that looks like the image below labeled Pre-correction:

    enter image description here

    There is a button near the bottom of the control panel under the Centroids button labeled pg. It simply stands for "pixel group" and once the centroids have been calculated, this button can be pressed to bring up an additional panel to adjust flux depending on which pixel in the TPF file was the central pixel during a given frame. Frames 1 to ~114 used pixel # 1177 in the TPF file. Notice the first portion in blue in the pre-corrected lightcurve has a lower average flux than the rest of the curve. This area was adjusted up by a value of 400 using the pixel group variable for pixel 1177. Pixel 1325 comprised primarily of the green area of flux in the latter part of the curve and was also adjusted up slightly. Several outliers remained in the final post-corrected lightcurve, but they are minimal and future efforts will make it easy to eliminate these outliers.

    After adjusting the colorful quadrant variables, subtracting the background flux (with a 3x multiplier) and adding values with the pixel group variables, the extracted curve looked well balanced and ready for exporting.

    CSV file for this extraction

    An AKO view of this file

    I could have also eliminated the need for the quad variables in this case and just used the pixel group variables, but the number of outliers was about the same.

    I'd like to apply the centroids in an alternate manner in the future to adust the final flux, rather than these quad variables. There are a couple methods that I know of, but don't have the know how yet, so building this tool sort of forces me to learn new things all the time.

    note: Currently, all released lightcurves for the 9 day Engineering Test do not include the first ~114 datapoints, or about 2 1/3 days. In theory, there could be a stray transit in any one of these files that indicate a planet with a period greater than 9 days.

    Posted

  • Dolorous_Edd by Dolorous_Edd

    Great tool Troy!

    There are issues with using this tool in Firefox. I use Chrome, and I haven't tried any other browsers. Use at your own risk.

    (Never thought that I would say so!) but I would actually recommend Internet Explorer 10 it proved itself to be MUCH more reliable and faster than Chrome

    In this case

    Posted

  • troyw by troyw

    Awesome, thanks for the tip Ivan! I hope someone besides me finds this tool useful 😃

    Posted

  • Martti_Holst_Kristiansen by Martti_Holst_Kristiansen

    Hi Troy,

    I have not had the time to look at your tool yet, but I just want to let you know that all your work is appreciated and that I am looking forward to trying it out as soon as possible!

    Posted

  • troyw by troyw

    I need to come with something better for correcting the lightcurves in C1. I'm not coming out with good results using this method....

    Posted

  • ajamyajax by ajamyajax in response to troyw's comment.

    Any new planetary data from our friends here that PH'ers could look through? Just wondered if you looked into this also.

    http://archive.eso.org/cms.html

    Edit: I remembered seeing this story which looked promising:

    "New Exoplanet-hunting Telescopes on Paranal"

    NGTS facility achieves first light

    14 January 2015

    http://www.eso.org/public/news/eso1502/

    (but fyi this web page has lots of video and ads, and could be slow to view)

    "The Next-Generation Transit Survey (NGTS for short) — located at the European Southern Observatory's (ESO) Paranal Observatory — is designed to seek out planets two to eight times the diameter of Earth as they pass in front of their stars..."

    from: "New Telescope in Chile Now Searching for Alien Planets"

    http://www.space.com/28246-alien-planet-searching-telescope-online.html

    Posted

  • zoo3hans by zoo3hans

    Yes, this seems to be an exiciting project!

    Posted

  • Martti_Holst_Kristiansen by Martti_Holst_Kristiansen

    I found that it was too time consuming to divide the text manually so I searched the net for other solutions and found a text splitting HTML-program. It makes the process much easier and does the job really quickly and efficiently. I have attached it here and hope it helps.

    Get Log Splitter

    Posted