Kubernetes logoI recently built a machine learning model, trained it, and explored the implications of deploying it using KubernetesMachine learning trains a program to recognize patterns in data so when new data is provided, it can make predictions based on what it’s learned. Kubernetes is a container orchestrator that automates the deploying and scaling of containers. Packaging a machine learning program and deploying it on Kubernetes has the potential to help our customers with their increasingly complex machine learning needs.

First, I determined what I wanted to accomplish with machine learning, the tools I needed, and how to put them together.  Teaching a machine how to recognize imagery is fascinating to me, so I focused on image recognition. I found the PlanesNet dataset on Kaggle — a dataset focused on recognizing planes in aerial imagery.  The dataset would be simple enough to implement but have good potential for further exploration.

To build a proof of concept I used TensorFlow, TFLearn, and Docker.  TensorFlow is an open-source machine learning library and TFLearn is a higher-level TensorFlow API which aids in writing less code to get started.  TFLearn is a Python library, so I wrote my proof of concept in Python.  Docker was used to package the program into a container to run on Kubernetes.

Machine Learning

planes data set

The PlanesNet dataset has 32,000 color images, each at 20px by 20px.  There are 8,000 images classified as “plane” and 24,000 images classified as “no-plane.” Reading in the dataset and preparing it for processing was straightforward using Python libraries like Pillow and NumPy.  Then I broke the dataset up into a training set and a testing set using the train_test_split function in the sklearn Python library.  The training set was used to build the model weights while the testing set validated how well the model was trained.

Neural Network

One of the most complex parts was how to design my simple neural network. A neural network is broken into layers starting with the input layer, then one or more hidden layers, ending with the output layer.

Creating the input layer, I defined its shape, preprocessing, and augmentation. The shape of the PlanesNet data is a 20px by 20px by 3 colors matrix. The preprocessing allowed me to tweak preparing the data for both training and testing while the augmentation allowed me to perform operations (like flipping or rotating images) on the data while training.

Because I was working with imagery, I chose to use two convolutional hidden layers before arriving at the output layer. Convolutional layers perform a set of calculations on each pixel and its surrounding pixels, attempting to understand features (i.e., lines, curves) of an image. After my hidden layers, I introduced a dropout percentage rate, which helps to reduce overfitting.

Next, I added a readout layer which brought my neural network down to the number of expected outputs.  Finally, I added a regression layer to specify a loss function, optimizer, and learning rate.  With this network, I could now train it over multiple iterations, called epochs, until I reached the desired level of prediction accuracy.

diagram of a simple neural network

Simple Neural Network

Using this model, I could now predict if an image was of an airplane. I loaded the image, called the predict function and viewed the output.  The output gave me a percentage of likelihood that the image was an airplane.

Exploring Scaling

Now that I had my simple neural network for training and predicting, I packaged it into a Docker container so it could be run on more than my single computer.  As I examined the details of deploying to Kubernetes, two things quickly became apparent.  First, my simple neural network would not train across multiple container instances.  Second, my prediction program would scale well if I wrapped it in a web API.

For my simple neural network, I had not implemented anything to split a dataset, train multiple containers, and bring the results back together.  Therefore running multiple instances of my container using Kubernetes would only provide the benefit of choosing the container with the highest accuracy model. (The dataset splitting process along with the input augmentation causes each container to have different accuracies.)  To have multiple containers which coordinate learning in my simple neural network would require further design.

diagram of the container model

My container’s prediction program executes via the command line but wrapping it in a web API endpoint it would make it easier to use.  Since each instance of the container has the trained model, Kubernetes could scale up or down the number of running instances of my container to meet the demand of the web API endpoint.  Kubernetes also provides a method for rolling out updates to my container if I further train my network model.

Conclusion

This was an excellent exercise in building a machine learning model, training it to predict an airplane from aerial imagery, and deploying it on Kubernetes. Additional applications could include expanding the dataset to include different angles of planes along with recognizing various specific types of planes. It would also be beneficial to re-design the neural network to benefit from Kubernetes scaling to the training needs. Using Kubernetes to deploy a prediction API-based on a trained model is both beneficial and practical today.

Azure Web Apps Background

I’ve been working with Azure Web Apps for a long time. Before the launch of Azure Web Apps for Containers (or even Azure Web App on Linux), these web apps ran on Windows Virtual Machines managed by Microsoft. This meant that any workload running behind IIS (i.e., ASP.Net) would run without hiccups — but that was not the case with workloads which preferred Linux over Windows (i.e., Drupal).

Furthermore, the Azure Web Apps that ran on Windows were not customizable. This meant that if your website required a custom tool to work properly, chances are it was not going to work on an Azure Web App, and you’d need to deploy a full-blown IaaS Virtual Machine. There was also a strict lockdown regarding tools and language runtime versions that you couldn’t change. So, if you wanted the latest bleeding-edge language runtime, you weren’t gonna get it.

Azure Web Apps for Containers: Drum Roll

Last year, Microsoft released the Azure Web Apps for Containers or Linux App Service plan offering to the public. This meant we could build a custom Docker image containing all the binaries and files, and then deploy it on the PaaS offering. After working with the product for some time, I was like..

The product was excellent, and it was clear that it had potential. Some of  the benefits:

  • Ability to use a custom Docker image to run the Web App
  • Zero headaches from managing Docker containers
  • The benefits of Azure Web App on Windows like Backups, Kudu, Deployment Slots, Autoscaling (Scale up & Scale out), etc.

Suddenly, running workloads that preferred Linux or required custom binaries became extremely easy.

The Architecture

Compared to Azure Web App on Windows, the architecture implemented in Azure Web App for Containers is different.

diagram of Azure web apps architecture

Each of the above Web Apps is strictly locked down with minimal possibility of modification. Furthermore, the backend storage was based on Network File Shares which means that even if you don’t want any storage (like in cases when your app simply reads data from the database and displays it back), the app would still perform slowly.

diagram of Azure web apps architecture

The major difference is that the Kudu/SCM site runs in a separate container from the actual web app. Both containers are connected to each other with a private network. In this case, each App Service Plan is deployed on a separate Virtual Machine and all the plumbing is managed by Microsoft. The benefits of this approach are:

  • Better isolation. If your Kudu is experiencing issues, it reduces the chance of taking down your actual website.
  • Ability to customize the actual web app container running the website.
  • Better resource utilization

Stay tuned for the next part in which I would be discussing the various options related to Storage which are available in Azure Web App for Containers and their trade-offs.

Happy holidays!

Containers are, for good reason, getting a lot of attention.  For the cost of having to manage some complexity, they provide a unique level of flexibility, ability to scale, run software across cloud and on-premises environment…the list of benefits can go on and on.  And usually when you hear about containers in the technical press, they’re included in an overarching story about an organization that moved to some highly scalable, microservices-based architecture to meet their ridiculous capacity demands (Netflix, Google, etc.).

At the most basic level, however, containers are about being able to streamline the process of installing and running software. In fact, the fundamental concepts behind containers map almost one-to-one with what’s been traditionally required to install a piece of software on your laptop: Read More…

The rapid growth of the Internet of Things (IoT) is certainly exciting for government agencies, but it also brings many challenges (namely security and devising a clear strategy for integrating a still-evolving technology).

Last night’s #AzureGov meetup was a great evening of networking, demos, use cases and best practices for government agencies looking to quickly and securely deploy IoT.

Last night featured two speakers from Docker Public SectorChris Cyrus, vice president, and Andrew Weiss, lead engineer  –  who joined the Microsoft Azure Government team to discuss how containers are changing the way software is built and procured in government. They presented demos of Docker Containers deploying IoT in the cloud and gave insights into best practices and customer use cases for government IoT.

In case you missed it, @aisteam was there and livestreamed the event. (Videos below.) Read More…

With the explosion of new sensors and service offerings producing geospatial telemetry, there’s an ever-increasing need for tools to gain business insights from this data. One of the premier tools for this in the geospatial domain is GeoServer.

Fully open-source and free to use, GeoServer provides Open Geospatial Consortium (OGC) web service interfaces to rendering images or complete metadata in most common geospatial interchange formats. In a consulting capacity, Applied Information Sciences has leveraged Geoserver with great success, allowing us to deploy a complete software stack in minutes instead days or weeks. In this post I’ll give an overview of the DevOps practices we’ve applied to enable this capability, as well as a brief overview of the supporting technologies. Read More…

Companies are adopting Docker containers at a remarkable pace and for a good reason – Docker containers are turning out to be key enablers for a micro-services based architecture.

As a quick recap, Docker containers are:

  • Encapsulated, deployable components that can run as isolated instances
  • Small in size with a fast boot-up time
  • Include tools that enable containerized application images to be easily moved across the public cloud and on-premises
  • Capable of applying limits on physical resources consumed by any given application

Given the popularity of Docker containers, it should come as no surprise that the Azure platform already provides first-class support for a container hosting solution, in the form of Azure Container Service (ACS). ACS makes it simple to create a cluster of Virtual Machines that can run containerized applications. ACS relies on popular open-source tools – with Docker as the container format, and a choice of Marathon, DC/OS, Docker Swarm and Kubernetes for orchestration and scheduling, etc. All this makes it possible to easily run containerized workloads on Azure in a portable manner.

But the Docker containerization story on Azure does not stop here.

It is also being weaved more and more into existing PaaS offerings, including Azure Batch, Azure App Service and Azure Service Fabric. Let’s briefly review the latest developments to see how Docker integrates with Azure PaaS: Read More…

These disciplines can play a significant role in building stable release processes that help ensure project milestones are met.

Continuous Integration (CI) and Continuous Delivery (DC) are rapidly becoming an integral part of software development. These disciplines can play a significant role in building stable release processes that help ensure project milestones are met. And in addition to simply performing compilation tasks, CI systems can be extended to execute unit testing, functional testing, UI testing, and many other tasks. This walkthrough demonstrates the creation of a simple CI/CD deployment pipeline with an integrated unit test.

There are many ways of implementing CI/CD, but for this blog, I will use Jenkins and GiHub to deploy the simple CI/CD pipeline. A Docker container will be used to host the application.  The GitHub repository hosts the application including a Dockerfile for creating an application node. Jenkins is configured with GitHub and Docker Plugin. Read More…

In this video blog, AIS’ CTO Vishwas Lele walks us through provisioning a Docker Swarm cluster using the Azure Container Service (ACS). Docker Swarm is a native clustering technology for Docker containers, which allows a pool of underlying Docker Hosts to appear as a single virtual Docker host. Containers can then be provisioned through the standard Docker API. The Azure Container service takes care of provisioning the underlying Docker host virtual machines, installing the required software (Docker plus  Docker host), and configuring the cluster. Once the cluster is provisioned, Vishwas demonstrates connecting to the master node to spin up containers on the cluster which hosts an ASP.NET application.

You might also like: 

The Seamless Hybrid Cloud

Enterprise #DevOps: A Service Catalog Driven Approach

Windows Azure Planning: Moving an Application into Production

27525399 - open window on white wall and the cloudy skyModern cloud computing offers enterprises unprecedented opportunities to manage their IT infrastructure and applications with agility, resiliency, and security, while at the same time realizing significant cost savings. The ability to rapidly scale up and down in the cloud opens countless doors of possibility to use compute and storage resources in innovative ways that were not previously feasible.

But getting to the cloud and managing both cloud and on-premises resources can be a daunting challenge. As a recent Gartner article explains, a Cloud Strategy is a must for organizations. That’s where we at AIS can help – we have years of experience and successes working with enterprises to develop a cloud strategy. We have the resources and expertise to then plan and execute, leveraging the latest technologies and best practices.

Read More…