Uploading Objects in Google Cloud Storage
Command Line
GCP provides Google Cloud CLI to interact with Google Cloud Storage through the command line. Google Cloud CLI includes the gcloud, gsutil and bq command-line tools. Here, I will use gsutil command tool to upload the object. For illustration, I am using sample.txt file. The file contains ‘This is a sample text’ as data.
Step 1: Use the below command to upload the object to gcs:
gsutil cp [LOCAL_FILE] gs://[BUCKET_NAME]/
The above command will upload LOCAL_FILE to the bucket named BUCKET_NAME.
Example: I have created a bucket named gfgbucket and uploaded sample.txt as below
gsutil cp [sample.txt] gs://[gfgbucket]/
Output:
Copying file://sample.txt [Content-Type=text/plain]...
/ [1 files][ 22.0 B/ 22.0 B]
Operation completed over 1 objects/22.0 B.
Goto the bucket from the cloud console to see our sample.txt as below
Object upload
API
You can access Google Cloud either through client libraries or REST APIs. Here I will illustrate how to use both the methods.
Client Libraries:
Client libraries are inbuilt libraries in the programming languages to access the Google Cloud Storage. Currently, GCP provides client libraries in C++, C#, Go, Java, Node.js, PHP, Python and Ruby. Below is the sample code in Java and Python using Google Client Library.
Step 2: Write the code in any of the following languages
Java
import com.google.cloud.storage.Blob; import com.google.cloud.storage.Bucket; import com.google.cloud.storage.Storage; import com.google.cloud.storage.StorageOptions; import java.nio.file.Path; import java.nio.file.Paths; public class GCSUploader { public static void main(String[] args) { // Replace these values with your actual GCP project ID and bucket name String projectId = "your-project-id"; String bucketName = "your-bucket-name"; // Path to the local file you want to upload String localFilePath = "path/to/local/file.txt"; // Destination object name in the Cloud Storage bucket String objectName = "your-object-name"; // Create a Storage client Storage storage = StorageOptions.newBuilder().setProjectId(projectId).build().getService(); // Get a reference to the target bucket Bucket bucket = storage.get(bucketName); // Get a Blob reference and upload the file Blob blob = bucket.create(objectName, Paths.get(localFilePath)); System.out.println("File uploaded to Cloud Storage: " + blob.getName()); } } |
Python3
# Importing the Google Cloud client library from google.cloud import storage # Initializing the client client = storage.Client() # Getting the bucket bucket = client.get_bucket('[BUCKET_NAME]') # Creating a blob (object) in the bucket blob = bucket.blob('[OBJECT_NAME]') local_file_path = 'path/to/local/file.txt' # Uploading the local file to the blob blob.upload_from_filename(local_file_path) print(f'File uploaded to Cloud Storage: {object_name}') |
Output:
File uploaded to Cloud Storage: sample.txt
REST API:
Through REST API, you can directly make HTTP request to Google Cloud Storage.
Step 3: Use the following cURL command to make HTTP request:
curl -X POST --data-binary @[LOCAL_FILE] -H
"Authorization: Bearer [YOUR_ACCESS_TOKEN]" -H
"Content-Type: [CONTENT_TYPE]" "https://storage.googleapis.com/upload/storage/v1/b/[BUCKET_NAME]/o?uploadType=media&name=[OBJECT_NAME]
In the above command, LOCAL_FILE is the path to the local file you want to upload, YOUR_ACCESS_TOKEN is authorization token that you get from your cloud service provider, OBJECT_NAME is the name you want to give to the object in your Google Cloud Storage Bucket.
Example: Let us upload sample.txt as sample2 to the bucket as below
curl -X POST --data-binary @sample.txt \
-H "Authorization: Bearer [YOUR_ACCESS_TOKEN]" \
"https://www.googleapis.com/upload/storage/v1/b/gfgbucket/o?uploadType=media&name=sample2"
Output:
{
"kind": "storage#object",
"id": "gfgbucket/sample2/1702468444308874",
"selfLink": "https://www.googleapis.com/storage/v1/b/gfgbucket/o/sample2",
"mediaLink": "https://www.googleapis.com/download/storage/v1/b/gfgbucket/o/sample2?generation=1702468444308874&alt=media",
"name": "sample2",
"bucket": "gfgbucket",
"generation": "1702468444308874",
"metageneration": "1",
"contentType": "text/plain",
"storageClass": "STANDARD",
"size": "22",
"md5Hash": "kzOPKF76QRKlpptuJ0TT0w==",
"crc32c": "ru17ew==",
"etag": "CIqL9ZatjIMDEAE=",
"timeCreated": "2023-12-13T11:54:04.352Z",
"updated": "2023-12-13T11:54:04.352Z",
"timeStorageClassUpdated": "2023-12-13T11:54:04.352Z"
}
Uploading and Downloading Objects in Google Cloud Storage: Command Line and API
Cloud storage refers to storing your data in remote locations i.e., the cloud. Cloud Storage became increasingly popular in the information era. With tons of data in hand, storing them in the cloud is very affordable. By storing the data in the cloud, we can access the data at any time irrespective of the device and the location.