Advanced

Using Spark DataFrames for large scale data science

Posted by bob on · Post a comment
Opensource.com
Mail this story
Print this story

When we first open sourced Spark, we aimed to provide a simple API for distributed data processing in general-purpose programming languages (Java, Python, Scala). Spark enabled distributed data processing through functional transformations on distributed collections of data (RDDs). This was an incredibly powerful API—tasks that used to take thousands of lines of code to express could be reduced to dozens. read more

Full Story

  Nav
» Read more about: Groups: Python; Story Type: News Story

» Get new stories by e-mail: nightly or as they are published

« Return to the newswire homepage

  Related stories
» More like this

Comments

No comments yet. Be the first.

Post a comment

A free account is needed to post. Write your comment here first: your text is kept in this browser while you sign up or log in, and is waiting for you when you come back.

Your message:
|||||
10
Markdown: **bold**, *italic*, [text](https://link), - lists, > quotes, $x^2$ for math. Formatting help

Sign up and post (it is free)  or  Log in and post