docs.calitp.org
Your uploaded table destination should always be the `uploaded_data` dataset in BigQuery (e.g. the `destination_table` name should always have the format `uploaded_data.your_new_table_name`).
Adding Ad-Hoc Data to the Warehouse — Cal-ITP Data Services
If you are working with a data source that needs to be updated regularly (like data that should be ingested/updated daily, weekly, or monthly), you should follow the general data pipeline architecture …
Transit Database (Airtable) — Cal-ITP Data Services
We ingest data from Airtable into the Cal-ITP data warehouse. For an overview of the data ingest process/architecture, see the pipeline architecture documentation.
Architecture Overview — Cal-ITP Data Services
Data pipelines to ingest specific types of data (ex. GTFS Schedule, Payments, etc.) Outside of this documentation, several READMEs cover initial development environment setup for new users.
Data Management — Cal-ITP Data Services
It is designed to make reading and writing data frames more efficient, as well as to make sharing data across languages easier. Just like Parquet, Feather is also capable of passing dataframes between …
Introduction to the Warehouse — Cal-ITP Data Services
The section serves as an introduction to the Cal-ITP warehouse through conventions, best practices, and tips and tricks to better understand the Cal-ITP data warehouse.
Data Services Documentation — Cal-ITP Data Services
Data Services Documentation # Welcome to Cal-ITP’s data services documentation! This resource serves as an opportunity to learn more about how the data services team works. Use the links below …
Open Data Intake | data-analyses - docs.calitp.org
This dataset compiles all the route and stop information for all CA transit operators and provides it in a geospatial format. It also compiles all the GTFS real-time vehicle positions data, processes it into a …
Cal-ITP Project Information — Cal-ITP Data Services
Caltrans’s Division of Data & Digital Services directly supports Cal-ITP by providing subject matter expert knowledge, analyzing data, and coordinating the interaction between CalSTA, Caltrans, …
Common Content — Cal-ITP Data Services
On this page you can find some of the common content types used in the Cal-ITP Data Services Documentation. Although the ecosystem we use, Jupyter Book, allows flexibility, the pages in our …