Open Distro for Elasticsearch is Amazon's move to show it's pro-open source
Amazon Web Services (AWS) just launched Open Distro for Elasticsearch. This program takes AWS's recent proprietary additions to Elasticsearch and open sources them under the Apache 2.0 license. That sounds good.
Shay Banon, co-founder of Elasticsearch and CEO of Elastic, its parent company, sees Amazon as forking and redistributing rebundled Elasticsearch for its own purposes: "From various vendors, to large Chinese entities, to now, Amazon. There was always a 'reason,' at times masked with fake altruism or benevolence" for "open-sourcing" their own Elasticsearch forks.
AWS claims this isn't so. "AWS. Open Distro for Elasticsearch leverages the open source code for Elasticsearch and Kibana. This is not a fork; we will continue to send our contributions and patches upstream to advance these projects," wrote Jeff Barr, AWS Chief Evangelist.
AWS VP of Cloud Architecture Strategy Adrian Cockcroft claimed: "When AWS launches a service based on an open-source project, we are making a long-term commitment to support our customers. We contribute bug fixes, security, scalability, performance, and feature enhancements back to the community. For example, we have been a significant contributor to Apache Lucene, which powers Amazon Elasticsearch Service."
That's not how others see it. AWS has been getting hammered recently for being an open-source user, which doesn't give back to the open-source communities. Open-source companies such as MariaDB, Redis, and MongoDB have all accused AWS, such as MariaDB CEO Michael Howard, of strip-mining open source.
Cockcroft is unsympathetic to Redis and MongoDB, which responded to AWS's competition by adding on anti-Software-as-a-Service licenses to their programs. He said, "We are seeing … examples where open source maintainers are muddying the waters between the open source community and the proprietary code they create to monetize the open source. At AWS, we believe that maintainers of an open-source project have a responsibility to ensure that the primary open source distribution remains open and free of proprietary code so that the community can build on the project freely, and the distribution does not advantage any one company over another."
As for the project itself, ElasticSearch is a search engine for structured and unstructured data. It's largely used for log analytics, full-text search, and business analytics. Open Distro for ElasticSearch is opening sourcing a set of advanced security, event monitoring and alerting, performance analysis, and SQL query features.
In more detail, these include:
- Security: This plugin that supports node-to-node encryption, five types of authentication (basic, Active Directory, LDAP, Kerberos, and SAML), role-based access controls at multiple levels (clusters, indices, documents, and fields), audit logging, and cross-cluster search so that any node in a cluster can run search requests across other nodes in the cluster.
- Event Monitoring and Alerting: This feature notifies you when data from one or more Elasticsearch indices meets certain conditions. You could, for example, notify a Slack channel if an application logs more than five HTTP 503 errors in an hour. Monitoring is based on jobs that run on a defined schedule, checking indices against trigger conditions, and raising alerts when a condition has been triggered.
- Deep Performance Analysis: This is a REST API that allows you to query a long list of performance metrics for your cluster. You can access the metrics programmatically or you can visualize them using perf top and other perf tools
- SQL Support: This feature allows you to query your cluster using SQL statements. It is an improved version of the elasticsearch-sql plugin, and supports a rich set of statements.
Besides providing Open Distro Source code repo, the program is available as RPM packages and Docker containers, with separate downloads for the SQL JDBC and the PerfTop CLI. You can run this code on your laptop, datacenter, or, of course, the cloud.
Banon is unimpressed. "Our commercial code has been an 'inspiration' for others, it has been bluntly copied by various companies, and even found its way back to certain distributions or forks, like the freshly minted Amazon one, sadly, painfully, with critical bugs.
Must read
- Amazon may be building an Alexa home robot (CNET)
- Amazon Web Services: A cheat sheet (TechRepublic)
Further, Banon continued, "Our brand has been used and abused, hijacked, and misrepresented many times. Companies have falsely claimed that they work in collaboration with our company, topically Amazon. We did not let it distract us, we kept on building great products and communities, that users love. Dilution of focus is the enemy of a company, and we never let it affect us. It is you that matter, our users, not the noise around it."
Matt Asay, head of Adobe developer ecosystem, thinks we've been too hard on AWS:
It has become so easy to point fingers at AWS over its open source contributions (or lack thereof). First we slammed them for not contributing enough. Now we're slamming them for potentially contributing too much. Lost in all this is the user, who just wants great software. Will AWS contributions to Elasticsearch significantly improve the code available for and around that project? It seems so, yes. Will that same code undermine Elastic NV's ability to monopolize revenues around the project? The answer to this, too, seems to be "yes." But if our focus is on the long-term viability of Elasticsearch, and not necessarily the long-term viability of any particular vendor thereof, is this a bad thing? I would argue that we're fixated on the wrong thing (vendors and their business models) and not nearly enough on the more important matter of open source sustainability. The more contributors to a project, in general, the better. We should welcome AWS' increased commitments to open source, not disparage them.
Is AWS being more of a friend or an enemy to open source? We'll see. This is certainly a new twist to its long and complicated relationship with the open-source community.