From Scala and Kubernetes to Google BigQueryML and Databricks' MLFlow, in this month's Big Data round-up we are sharing some of the most recent posts and announcements that caught the eye of our Big Data specialists.
Big Data
- Data Lakes Keep Rising While Hadoop Sinks [Database Trends & Applications]
- SOLR continues to lose popularity year after year; it’s well behind Elasticsearch which is now the most popular search solution. [DB Engines]
- “Hadoop & HDFS-based” solutions are starting to lose market traction, perhaps becoming “legacies”… Many vendors and clients are losing interest in Hadoop/HDFS solutions. [Silicon Angle]
- Serializing, Parsing and Pickling Data in Scala [GitHub]
- Issue #140 Solved! Build and Test Spark Against Scala 2.12 [Apache]
- Doing Without Databases in the 21st Century [codeburst]
Artificial Intelligence & Machine Learning
- How AI will shape the future. [GQ Magazine]
- 8 Game changing data technologies. [Database Trends & Applications]
- Why do you think most of the companies want to adopt AI and Machine Learning? Definitely not to reduce labour costs. [CIO Dive]
- Automated Machine Learning for Structured Data on Spark [TransmogrifAI]
- Know how to count the number of people who are heading “in” or “out” of a department store in real-time. [pyimagesearch]
- Databricks' MLFlow - A Platform for the complete Machine Learning lifecycle [miflow]
- [new] Google BigQueryML - Machine Learning to bridge the gap between data processing/data ops and machine learning [Google]
- A Scala library for machine-learning ETL operations [GitHub]
DevOps
- In terms of automation and DevOps, Docker+Kubernetes are now becoming the de-facto standard for container orchestration. Here’s a nice intro tutorial to Kubernetes. [okigiveup]
- Dynamic Scaling for Computer Vision with Pub/Sub Messaging and Docker [MapR]
- Knative- Kubernetes-based platform to build, deploy, and manage modern serverless workloads [GitHub]
Digital Transformation
- According to IDC, Retail spending on Digital Transformation is growing at a compound annual growth rate of 20.2%, which is faster than overall digital transformation spending. [IDC]
- Why 71% of organizations will spend more on data in the next five years. [Tech Republic]
- Biggest barriers to digital effectiveness [The Financial Brand]
If you would like to find out more about how Big Data could help you make the most out of your data while enabling you to open your digital horizons, do give us a call at +44 (0)203 475 7980 or email us at Salesforce@coforge.com
Other useful links:
Survey Report: The State of Big Data in the UK 2017/2018
The Big Data 'Problem'
Banking and The Internet of Things
