Best Way To Upload A 59 GB Parquet Dataset From Google Drive To Kaggle?

​

I have a ~59 GB Parquet dataset stored in Google Drive, and I want to use it for ML training on Kaggle GPUs.

My folder structure is:

My Drive/

└── DATASET/

└── data/

└── processed/

└── era5_land_preprocessed/

├── train/ → 60 Parquet files → 44.34 GB

├── validation/ → 12 Parquet files → 8.83 GB

└── test/ → 8 Parquet files → 5.82 GB

Total: 80 Parquet files, ~58.99 GB

I don’t want to download the dataset to my local computer. I was planning to use Google Colab + Kaggle API to transfer the files directly from Google Drive to a Kaggle Dataset.

My questions are:

1. Is uploading a ~59 GB dataset to Kaggle this way reliable?

2. Is Google Drive → Colab → Kaggle API the best approach?

3. Should I create one Kaggle dataset containing “train/”, “validation/”, and “test/”, or split them into multiple Kaggle datasets?

4. Are there any problems with uploading large Parquet files to Kaggle?

5. Is there a better/faster method for transferring this amount of data to Kaggle?

The dataset will be used for climate/weather ML forecasting, and I need to train models using Kaggle’s GPUs.

Any advice from people who have handled 50+ GB datasets on Kaggle would be appreciated.