Try our Chrome extension
Easily add the current web-page from your browser directly into your changedetection.io tool, more great features coming soon!
Chrome Webstore
Changedetection.io needs your support!
You can help us by supporting changedetection.io on these platforms;
- Rate us at AlternativeTo.net
- Star us on GitHub
- Follow us at Twitter/X
- G2 Software reviews
- Check us out on LinkedIn
- And tell your friends and colleagues :)
The more popular changedetection.io is, the more time we can dedicate to adding amazing features!
Many thanks :)
changedetection.io team
Not yet seconds ago.
False
Not yet seconds ago
3 hours agoGoto single snapshot
Skip to content
Navigation Menu
Sign in Appearance settings
* Platform
+ AI CODE CREATION
o GitHub Copilot Write better code with AI
o GitHub Copilot app Direct agents from issue to merge
o MCP Registry Integrate external tools
+ DEVELOPER WORKFLOWS
o Actions Automate any workflow
o Codespaces Instant dev environments
o Issues Plan and track work
o Code Review Manage code changes
o Code Quality Enforce quality at merge
+ APPLICATION SECURITY
o GitHub Advanced Security Find and fix vulnerabilities
o Code security Secure your code as you build
o Secret protection Stop leaks before they start
+ EXPLORE
o Why GitHub
o Documentation
o Blog
o Changelog
o Marketplace
View all features
* Solutions
+ BY COMPANY SIZE
o Enterprises
o Small and medium teams
o Startups
o Nonprofits
+ BY USE CASE
o App Modernization
o DevSecOps
o DevOps
o CI/CD
o View all use cases
+ BY INDUSTRY
o Healthcare
o Financial services
o Manufacturing
o Government
o View all industries
View all solutions
* Resources
+ EXPLORE BY TOPIC
o AI
o Software Development
o DevOps
o Security
o View all topics
+ EXPLORE BY TYPE
o Customer stories
o Events & webinars
o Ebooks & reports
o Business insights
o GitHub Skills
+ SUPPORT & SERVICES
o Documentation
o Customer support
o Community forum
o Trust center
o Partners
View all resources
* Open Source
+ COMMUNITY
o GitHub Sponsors Fund open source developers
+ PROGRAMS
o Security Lab
o Maintainer Community
o Accelerator
o GitHub Stars
o Archive Program
+ REPOSITORIES
o Topics
o Trending
o Collections
* Enterprise
+ ENTERPRISE SOLUTIONS
o Enterprise platform AI-powered developer platform
+ AVAILABLE ADD-ONS
o GitHub Advanced Security Enterprise-grade security features
o Copilot for Business Enterprise-grade AI features
o Premium Support Enterprise-grade 24/7 support
* Pricing
Type / to search
Sign in
Sign up Appearance settings
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
Uh oh!
There was an error while loading. Please reload this page.
apache / spark Public
* Notifications You must be signed in to change notification settings
* Fork 29.3k
* Star 43.8k
* Code
* Issues 41
* Pull requests 411
* Pull requests 409
* Actions
* Security and quality 0
* Insights
Additional navigation options
* Code
* Issues
* Pull requests
* Actions
* Security and quality
* Insights
master
Branches Tags
Go to file
Code
Open more actions menu
Folders and files
Name Name Last commit message Last commit date
Latest commit
History
49,531 Commits
49,531 Commits
49,536 Commits
49,536 Commits
.github .github
.mvn .mvn
R R
assembly assembly
bin bin
binder binder
build build
common common
conf conf
connector connector
core core
data data
dev dev
docs docs
examples examples
graphx graphx
hadoop-cloud hadoop-cloud
launcher launcher
licenses-binary licenses-binary
licenses licenses
mllib-local mllib-local
mllib mllib
project project
python python
repl repl
resource-managers resource-managers
sbin sbin
sql sql
streaming streaming
tools tools
udf/ worker udf/ worker
ui-test ui-test
.asf.yaml .asf.yaml
.gitattributes .gitattributes
.gitignore .gitignore
.nojekyll .nojekyll
.pre-commit-config.yaml .pre-commit-config.yaml
.sbtopts .sbtopts
AGENTS.md AGENTS.md
CLAUDE.md CLAUDE.md
CONTRIBUTING.md CONTRIBUTING.md
LICENSE LICENSE
LICENSE-binary LICENSE-binary
NOTICE NOTICE
NOTICE-binary NOTICE-binary
README.md README.md
SECURITY.md SECURITY.md
pom.xml pom.xml
pyproject.toml pyproject.toml
scalastyle-config.xml scalastyle-config.xml
View all files
Repository files navigation
*
* README
* Code of conduct
* Contributing
* Apache-2.0 license
* Apache-2.0 license
* Security
More items
Apache Spark
Spark is a unified analytics engine for large-scale data processing. It provides high-level APIs in Scala, Java, Python, and R (Deprecated), and an optimized engine that supports general computation graphs for data analysis. It also supports a rich set of higher-level tools including Spark SQL for SQL and DataFrames, pandas API on Spark for pandas workloads, MLlib for machine learning, GraphX for graph processing, and Structured Streaming for stream processing.
* Official version: https://spark.apache.org/
* Development version: https://apache.github.io/spark/
Online Documentation
You can find the latest Spark documentation, including a programming guide, on the project web page. This README file only contains basic setup instructions.
Build Pipeline Status
Branch Status
master
branch-4.x
branch-4.3
branch-4.2
branch-4.1
branch-4.0
branch-3.5
Building Spark
Spark is built using Apache Maven. To build Spark and its example programs, run:
./build/mvn -DskipTests clean package
(You do not need to do this if you downloaded a pre-built package.)
More detailed documentation is available from the project site, at "Building Spark".
For general development tips, including info on developing Spark using an IDE, see "Useful Developer Tools".
Interactive Scala Shell
The easiest way to start using Spark is through the Scala shell:
./bin/spark-shell
Try the following command, which should return 1,000,000,000:
scala> spark.range(1000 * 1000 * 1000).count()
Interactive Python Shell
Alternatively, if you prefer Python, you can use the Python shell:
./bin/pyspark
And run the following command, which should also return 1,000,000,000:
>>> spark.range(1000 * 1000 * 1000).count()
Example Programs
Spark also comes with several sample programs in the examples directory. To run one of them, use ./bin/run-example <class> [params]. For example:
./bin/run-example SparkPi
will run the Pi example locally.
You can set the MASTER environment variable when running examples to submit examples to a cluster. This can be spark:// URL, "yarn" to run on YARN, and "local" to run locally with one thread, or "local[N]" to run locally with N threads. You can also use an abbreviated class name if the class is in the examples package. For instance:
MASTER=spark://host:7077 ./bin/run-example SparkPi
Many of the example programs print usage help if no params are given.
Running Tests
Testing first requires building Spark. Once Spark is built, tests can be run using:
./dev/run-tests
Please see the guidance on how to run tests for a module, or individual tests.
There is also a Kubernetes integration test, see resource-managers/kubernetes/integration-tests/README.md
A Note About Hadoop Versions
Spark uses the Hadoop core library to talk to HDFS and other Hadoop-supported storage systems. Because the protocols have changed in different versions of Hadoop, you must build Spark against the same version that your cluster runs.
Please refer to the build documentation at "Specifying the Hadoop Version and Enabling YARN" for detailed guidance on building for a particular distribution of Hadoop, including building for particular Hive and Hive Thriftserver distributions.
Configuration
Please refer to the Configuration Guide in the online documentation for an overview on how to configure Spark.
Contributing
Please review the Contribution to Spark guide for information on how to get started contributing to the project.
About
Apache Spark - A unified analytics engine for large-scale data processing
spark.apache.org/
Topics
big-datajavajdbcpythonrscalasparksql
Resources
Readme
Apache-2.0, Apache-2.0 licenses found
Code of conduct
Code of conduct
Contributing
Contributing
Security policy
Security policy
Activity
Custom properties
Stars
43.8k stars
Watchers
2.0k watching
Forks
29.3k forks
Report repository
Releases
Packages
Used by
Contributors
Languages
Footer
© 2026 GitHub, Inc.
Footer navigation
* Terms
* Privacy
* Security
* Status
* Community
* Docs
* Contact
* Manage cookies
* Do not share my personal information
You can’t perform that action at this time.
Tip: Highlight text to share or add to ignore lists.
— Download difference patch
For now, Differences are performed on text, not graphically, only the latest screenshot is available.
Screenshot requires Playwright/WebDriver enabled