Software Engineer by Profession, passionate about Data Science and Machine Learning
Image for post
Image for post
Photo by Delia Giandeini on Unsplash

Working with TFRecords with an introduction to Protobufs

In this post, I’m going to discuss Tensorflow Records. Tensorflow recommends to store and read data in tfRecords format. It internally uses Protocol Buffers to serialize/deserialize the data and store them in bytes, as it takes less space to hold an ample amount of data and to transfer them as well.

Protobufs work with the predefined schemas, unlike JSON and XML. tfRecords has such schemas already available in Protofiles, and also the compiled code for many supported languages. I, in the post, am going to import such compiled code in Python and use them on my data.

This post needs a basic understanding of Protobufs, as they are the building blocks of TFRecords. I have also written a post earlier about working with Protobufs. In this post, I start by going over the predefined protos for tfRecords, using them in Python with some dos and don’ts. Then I’ve two demos where I’ll be making tfRecords from existing datasets. …

PROGRAMMING REST APIS

Parse a raw file or a file in HTML form-data

Image for post
Image for post
Image by Girija Shankar Behera

In this article we will explore how to handle file uploads in REST API frameworks. Earlier I have published a few articles related to REST APIs where we have covered all the Basics, and the Constraints to make an API Restful, and then a couple of hands-on (Part 1, Part 2 and Part 3).

This time, we are going to see something different. This does not really impact on the RestFul-ness of the API, but a useful feature to have while building an API system. i.e. handling of file uploads via an API request. This article is going to be short as there’s not much to explore in it. …

PROGRAMMING REST APIS

Complete the system of APIs by making them RESTful

Image for post
Image for post
Image by Girija Shankar Behera

In the REST API series, this is the fifth and the final post. The first two are the theoretical ones where I’ve not covered any implementation, but looked into what REST APIs are, how to use them, and the architectural constraints in REST. In the third and the fourth articles, I’ve been developing an API system for a Book Catalogue application using all the knowledge gained in the first two posts about REST. If you haven’t been to these posts, then please have a look and then come back here.

This post extends the development further. Specially, I’m going to update the serializers some more, as the API system till now is not yet RESTful. …

PROGRAMMING REST APIS

GET, POST, PUT, DELETE. and what not!!

Image for post
Image for post
Image by Girija Shankar Behera

This is the fourth post in the REST API series. In the first two posts, we explored the basics of REST APIs, how to use them, and the architectural constraints. In the previous post, we started with developing a Book Catalogue API system with the existing knowledge on REST APIs. For implementation details, I’ve used Django REST Framework because of its rich support for building web applications.

This article extends the development further. We were done with building the API endpoint to get a list of Books in the previous article. In this, we will continue with creating the APIs for the Author and Genre model. …

PROGRAMMING REST APIS

Easy Peasy with Django REST

Image for post
Image for post
Image by Girija Shankar Behera

Over the last two posts, I’m exploring REST APIs. We covered the basic details of REST APIs, the request and response, and what they are made up of, methods that can be used in REST. And then we also explored the architectural constraints of REST. We also discuss the GitHub use case which was quite interesting while learning. This time, I’m finally going to use all those knowledge and develop a API service using REST.

Let’s first design the whole application(not yet done).

PROGRAMMING REST APIS

Am I not RESTful? Why not?

Image for post
Image for post
Image by Girija Shankar Behera

In the previous post, we saw some of the basics of REST APIs, how a Request and a Response looks like, what they are made up of, the methods that can be used and some use cases. Those were all the basic stuff needed to create a REST API. But saying that, it does not ensure that the API service is still RESTful.

REpresentational State Transfer is an architectural style which defines certain guidelines or constraints while developing the API service. These constraints primarily focus on certain guidelines while designing the application, rather than going to the detailed implementation. …

PROGRAMMING REST APIS

How does Computers communicate?

Image for post
Image for post
Image by Girija Shankar Behera

Recently, during a job interview, I was asked questions about what REST APIs are. How do they work, and what differentiates them from SOAP? And, as easy as it might be for many, for me it’s something I’ve never spent much time on studying or understanding to their depth. It’s one of those topics I have never got a chance to explore. Never was required for any of my previous projects. And I feel it’s a very relatable topic if someone is in Software Development. Almost all of my friends and colleagues have good knowledge on them or have worked on them as part of their previous projects. …

Image for post
Image for post
Image by Girija Shankar Behera

In this post, I’m going to write about Google’s way of Serializing and Deserializing the data in a specific format. It was initially developed by Google Developers as their own internal mechanism of sending data over the wire, keeping an eye on enhancing network performance and usage. It was mainly designed to be simpler than XML, yet smaller and faster in its storage and transmission.

This post briefly covers the need to know things about Protobufs (Protocol Buffers). It also serves as a base for the next one as that will be about the Tensorflow Records. …

Image for post
Image for post

Sentinel-2 gives access to the better quality imageries of spatial resolution 10m, 20m and 60m. The satellite with its Multispectral Instrument (MSI) is capable of capturing 13 bands in the range of Visible/Near Infrared (VNIR) to Short Wave Infrared (SWIR). In this post, I’m working on accessing these imageries, filtering them out for a Region of Interest and exploring several tools/libraries used to download and perform operations on them.

This post is in continuation with the previous one. The previous post was accessing satellite imagery layers, accessing feature layers, querying and using raster functions on them. All of them were being done online without downloading. …

Image for post
Image for post

Satellite data provides much crucial and decisive information for a wide range of applications such as land cover change detection, mineral exploration, monitoring wildfires, disasters and various natural calamities. The greatest events on Earth can be traced back to the point of time when they were actually happening, using the Historical Satellite Images. They allow us to go back in time and see how Earth looked in the past. In the age of Big Data, there is no other conventional equipment that can offer such amount of real time data at a global level.

In the past decades, many Earth Observing Satellite missions have come into existence, each one with certain objectives, some were driven by the Government entities and some were driven commercially by various private firms. Sentinel Program is one such program developed by European Space Agencies, whose mission was to replace the older generation of Earth Observation Satellites, as most of them were nearing their retirement. The earlier Sentinel missions (Sentinel-1 & 2) were monitoring Land and Ocean. The later ones (Sentinel-3, 4, 5, & 5P) have focused more on the air quality data, ozone and UV radiation. Landsat Program is one of the longest running Satellite programs, managed jointly by NASA and U.S. Geological Survey. They have had 8 missions till date, of which 6 older ones (Landsat 1 — Landsat 6) have been discontinued and Landsat 7 and 8 are currently on their mission, continuously acquiring new imageries. The data collection from Landsat allows researchers to analyze and track land use, and land changes over the time caused due to climate change, urbanization, wildfires, etc. It is also being used for monitoring crops, vegetation and forestry. …

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store