Every premium and members-only resource for $9.99/month, as a no-commitment subscription. See the plans

PalateKit

Sign in
Menu

Request a resource

Describe the resource you are missing in one sentence. We will email you when it is published.

0 / 280

A page, a video or an image online.

Unlock PalateKit

Unlock everything

Plan

Paying means accepting our terms of sale.

Secure payment by Stripe. Receipts and invoices sent by email.

Welcome back

Sign in to install, download or save this resource. A free account is enough for free resources.

PySpark ETL best practices v1.0.0

Use this resource to write testable, readable PySpark ETL pipelines

New

  • 115 downloads
  • 243 likes
  • 130 saves

Free

Install with the CLI

In a terminal, from the folder you want:

npx --yes palatekit@latest install pyspark-etl-practices-rule

First time: connect the CLI with

npx --yes palatekit@latest login

Cursor structures each job around a base class that manages the SparkSession, separates config from argument parsing, composes processing into named steps with .transform(), and follows patterns for joins, window functions and Iceberg tables.

Mostly picked by

Updated November 2, 2026

Card for “PySpark ETL best practices”

Details

Works in
Install guides
Generates with
Your agent's model
Licence
CC0-1.0

Similar resources

View all

What you get

  • The .mdc file (in English): job structure, configuration and step composition
  • Examples of a base ETL class and a factory-based config

Good fit for

  • A Spark pipeline that must be tested locally
  • A data team that loads Iceberg tables

Not for

    Tags

    • pyspark
    • etl
    • spark
    • iceberg

    About

    You say

    “Turn this PySpark script into an ETL job with a base class and tests.”

    Card for “PySpark ETL best practices”

    Rule published by PatrickJS under the CC0-1.0 license, delivered in its original version.

    Show more

    How to use it

    1. Copy the .mdc file into your project's .cursor/rules/ folder
    2. Ask Cursor to write testable, readable PySpark ETL pipelines
    3. Adjust the rule to your conventions

    Content preview

    Files

    • LICENSE6.5 KB
    • NOTICE-PALATEKIT.md0.5 KB
    • pyspark-etl-practices-rule.mdc13.6 KB

    Origin and licence

    Listed unmodified

    Author
    PatrickJS
    Source
    github.com/PatrickJS/awesome-cursorrules/blob/b044f956f021b6e8877f16781bcfc466a6a120e9/rules/pyspark-etl-best-practices-cursorrules-prompt-file.mdc
    Licence
    CC0-1.0

    Notice

    « pyspark-etl-best-practices-cursorrules-prompt-file » (PatrickJS, CC0-1.0) livré dans sa version d'origine : fiche en français et en anglais par PalateKit.

    User reviews

    –

    No ratings yet

    Loading reviews…

    Delete your review?

    Your rating and comment will be removed from the page.