DevOps Newsletter – November 2019

Traffic Routing Patterns

Edge Proxy Server

• An edge proxy is a proxy server that sits on the edge of your infrastructure, providing access to internal services.
• You can think of an edge proxy as the “front door” to your internal service architecture—it allows clients on the internet to make requests to internal services you deploy.
• There are multiple open source edge proxies that have a robust feature set and community, so we don’t have to write and maintain our own edge proxy server.
• One of the most popular open source edge proxy servers is called Zuul and is built by Netflix.
• Zuul is an edge service that provides dynamic routing, monitoring, resiliency, security, and more.
• Zuul is packaged as a Java library.
• Services written in the Java framework Spring Boot can use an embedded Zuul service to provide edge-proxy functionality.

Key objectives of this chapter

• Edge Proxy Server
• API Gateway
• Rate Limiting
• Circuit Breaker
• Load Balancer
• Service Mesh
• Content Delivery Network
Continue Reading…

 

Courses
WA2675 Architecting Microservices with Kubernetes, Docker, and Continuous Integration
This two-day DevOps training class is loaded with practical real-world information. From the foremost Service-Oriented-Architecture/DevOps/Agile development training company, comes a course to move the dial on your organizations’ journey with this workshop. You will leave this course loaded with knowledge on the usage of this stack for Microservices. This class combines instructor-led and hands-on lab exercises.


 

Leading Practices for Microservice Logging

Logging Challenges

• (Micro-)Service Architectures mean that solutions are distributed across a network topology, with many small processes involved in handling a single request.
• Simple logging solutions generally don’t scale beyond a single process.
• How do we manage logs, which contain forensic information, when the requests have been distributed to many, potentially load-balanced, processes?

Leading Practices

• Correlate Requests with a Unique ID
• Include a Unique ID in the Response
• Send Logs to a Central Location
• Structure Your Log Data
• Add Context to Every Request
• Write Logs to Local Storage

Continue Reading…

 

Courses
WA2543 DevOps BootCamp
DevOps is the combination of cultural philosophies, practices, and tools that increases an organization’s ability to delivery applications and services at high velocity. Under a DevOps model, a development and operations teams are no longer “siloed”. Quality assurance team also becomes more tightly integrated with development and operations and throughout the application lifecycle. Web Age Solutions’ DevOps BootCamp course covers the DevOps principles and tools required for end-to-end implementation of DevOps.


 

Defining Data Engineering

What is Data Engineering

• Data engineering is a software engineering practice with a focus on design, development, and productionizing of data processing systems
• Data processing includes all the practical aspects of data handling, including:
• Data acquisition, transfer, transformation, and storage on-prem or in the cloud
• In many cases, data can be categorized as Big Data (see the slide’s notes)

Gartner’s Definition of Big Data
Gartner’s analyst Doug Laney defined three dimensions to data growth challenges: increasing volume (amount of data), velocity (speed of data in and out), and variety (range of data types and sources) In 2012, Gartner updated its definition as follows: “Big data are high volume, high velocity, and/or high variety information assets that require new forms of processing to enable enhanced decision making, insight discovery and process optimization.”

Volume
Data sizes accumulated in many organizations come to hundreds of terabytes, approaching the petabyte levels

Variety
Big Data comes in different formats as well as unformatted (unstructured) and various types like text, audio, voice, VoIP, images, video, e-mails, web traffic log files entries, sensor byte streams, etc.

Velocity
High traffic on-line banking web site can generate hundreds of TPS (transactions per second) each of which may be required to be subjected to fraud detection analysis in real or near-real time

Continue Reading…

 

Courses
WA2905 Data Engineering with Python
Data engineering is a software engineering practice with focus on design, development, and the productionizing of data processing systems. It includes all the practical aspects of data acquisition, transfer, transformation, and storage on-prem or in the cloud. This intensive hands-on training course teaches the students how to apply Python to the practical aspects of data engineering and introduces the students to the popular Python libraries used in the field, including NumPy, pandas, Matplotlib, scikit-learn, and Apache Spark.