The CRTS DR3 Archive

The Catalina Real-time Transient Survey is a collection of three telescopes which have operated over a period spanning more than a decade. The telescopes visit specific areas of the sky at defined intervals of time and take images. These images are then processed to derive photometric measurements. IUCAA is a host to the image archive. Some features of this image archive include:

  • A script that cuts-out the large field images into small images with more convenient footprint and manageable size.
  • A MySQL database with 8 billion records, which utilize the Hierarchical Triangular Mesh (HTM) to enable speedy searches based on the 2-d position on the celestial sphere.
  • A Virtual Observatory (Simple Image Access Protocol) compliant web service that allows all VO compatible tools to access these images.

The site may be accessed here.

The CZTI Data Management System

The Cadmium Zinc Telluride Image (CZTI) is one of five payloads onboard the Astrosat satellite, the first ever space telescope launched by India. The Payload Operations Center for this instrument is situated right here in IUCAA! So, in collaboration with them, we decided to do some complex database management system development. Features include:

  • Detailed track of every single processing pipeline parameter including the SVN checkout version.
  • A modern web based GUI for searching observations based on a number of input parameters.
  • Search results include options such as previewing history of the processing, information about multiple versions of data products etc.

In an ideally planned project, such a system would allow storage of detailed metadata, sufficient to construct any version of the data product. Thus, as new versions get created, one may dispose of these products knowing that should the need arise, the metadata can be used to reconstruct the product on demand.

Detection of Bars in Galaxies using Deep Learning

Galaxies come in a wide variety of shapes. Some galaxies exhibit a bar like structure. Now, it turns out that this bar's gravitational field causes interesting changes in how a galaxy evolves. Thus it is very important for us to classify galaxies as barred / unbarred. And how does one do that? Well, you stare at the image and your brain tells you it is a barred galaxy. Okay, let's do that for the millions of galaxies observed by various astronomical surveys. Sounds like a job for deep learning!

We worked with a group where we explored the use of Deep Learning algorithms, specifically Convolutional Neural Networks to train a machine in classifying galaxies as barred or unbarred. The algorithm does a fantastic job with an accuracy of 94%. More details here.

We built a web application to enable access to the neural network which can be found here.

The Kodaikanal Digitized Data Archive

The Kodaikanal Solar observatory is operational since more than a century. Dedicated observers have been at work since 1904, imaging the sun using photographic plates. Given that the sun exhibits periodical changes such as sun spots over periods spanning decades, such a data set becomes immensely important. To make the data amenable to the sophisticated analysis computerized routines available, they must be present in digitized form. The folks at Indian Institute of Astrophysics (IIA, which now operates the Kodaikanal Observatory), have carried out an immense undertaking, lasting several years, to digitize these data.

As a part of this project, we built an archive for them. We used modern web frameworks to design a data archive which would make data access easier for the scientific community. Novel features we included in this site include the following:

  • An interactive histogram based visualization, coded using D3.js which allows users to get a feel for the distribution of the data spanning 100+ years.
  • A time uniformity visualization which informs the user about the sampling uniformity along time, for a given search data set. This helps the user make informed decisions about whether the data is amenable for the nature of analysis intended.
  • A Virtual Solar Observatory compatible SOAP service layer. This will enable generic data scraping tools capable of speaking the VSO SOAP specification, to stumble across Kodaikanal digitized data.

You can visit the site at this link.

Adaptation of Alfresco ECM

Alfresco is an open source Enterprise Content Management System which enables easy management of unstructured documents. We explored the feasibility of adapting this system for the instrumentation lab based in IUCAA. Apart from team training and interfacing, the developer team also carried out the construction of some extensions to suit the needs of the lab. Specific features include exposing a document lock option in the GUI so that documents which have reached a final uneditable state can be marked; enforcement of security option which prevent nobody but project managers from deleting documents; and more.

The OAD Resources Archive

The Office of Astronomy for Development (OAD) is a joint project of the International Astronomical Union (IAU) and the South African National Research Foundation (NRF) with the support of the Department of Science and Technology (DST). The OAD primarily carries out its mission by funding and coordinating projects that use astronomy as a tool to address issues related to sustainable development. Since 2013, more than 120 projects have been funded through the annual Call for Proposals.

In the first part of the project, we helped the OAD adapt the Alfresco Enterprise Content Management system to manage their project sources. We then built an interface that enables users to search through materials produced by OAD's past funded projects. Users specify keywords in the primary search box. They can further narrow the search results using the filters for Project Title, Location, Year (of project implementation), and SDG (View the Sustainable Development Goals)

The interface may be from this link.

A Fully Updated Vizier Mirror

Vizier is a catalog hosting service created and maintained by the Strasbourg astronomical Data Centre. This service enables astronomers to share and access catalogs of all sizes. While many mirros of this service are hosted world wide, CDS remained the sole providers of the big data catalogs such as GAIA DR2 and SDSS DR 12. The teams at IUCAA and CDS worked together to setup the first fully updated Vizier mirror at IUCAA. Leveraging virtualization technologies such as containers, the process was carried out most efficiently.

The mirror is accessible from this link.

Services to the Biology Community

We are also working with colleages from Regional Centre of Biotechnology, at Faridabad to design data storage and dissemination solutions for large data sets gathered using complex instruments such as the new X-Ray Diffraction facility at the European Sycnhrotron Research Facility. Generally, biological data have been produced in personal labs and thus the need for a centralized archive with appropriate search and download services has been scarce. However, the situation in Astronomy has been the opposite. Astronomers often have had to work in far-off locations in facilities hard to construct using a single lab's resources.

We have also been assisting the scientists from the National Center for Cell Sciences (NCCS) in data transfers and storage from international facilities.

A Talk Request Management System

The Outreach wing of IUCAA caters to several requests every month from colleges, universities and institutes around the country. A typical request is initiated over e-mail followed by a manual management and coordination between speakers assigned and the people who arrange the programs. As a quick solution, a web application was developed by members in our team to help them with the management of these requests.

Dark Skies For All

The "Dark Skies For All" is an IAU 100 program which aims at bringing people together to sensitivize people around the world to the importance of preserving our dark skies. By reaching out to the media, government agencies, colleges and institutes, the dark sky ambassadors will try to achieve the goal of helping people preserve the beauty of the night skies.

The team at IUCAA has contributed in the end to end development of the dark sky ambassadors' portal as well as in the hosting this portal using on-premise hosting solutions. This can be accessed here.

The QuickDB Interface

This project was jointly done between National Astronomical Observatory of Japan (NAOJ) and the NKN team. NAOJ has developed a database system, currently christened as QuickDB, which uses pure Python to provide an SQL interface to a database and offers immensely fast processing speeds. By converting the SQL query into a MapReduce job which in turn is executed in parallel by a master-worker node configuration, QuickDB provides a fast alternative to traditional database systems.

The team at IUCAA helped build a scheduler system as well as a Health Keeping System. Work is underway to polish the complete interface. It will be made public in the coming year.

This site is hosted at IUCAA. Copyright © 2019.