Monday, 25 November 2019

js access azure blob

https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-nodejs#configure-your-storage-connection-string
https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/storage/storage-blob/samples

https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-nodejs#list-the-blobs-in-a-container

some key concepts i need to know before start:

BlobServiceClient  just like database connection, you must have it first. to create it you will need to get the connect string from azure port. just like database connection,.

ContainerClient  this just like your local disk, when you start to access your local disk, you need to tell which disk you want to access  e.g.  C:/ or D:/ ..... nothing special, it is just used to organise files. you also can consider this is different databases under your database.

You can get ContainerClient from BlobServiceClient by just tell the container name.

https://docs.microsoft.com/en-us/javascript/api/@azure/storage-blob/containerclient?view=azure-node-latest#getblobclient-string-

BlobClient this could be a file, could be a folder . this is the reference to your files and folders.   from ContainterClient, then you can get BlobClient. just like "dir" or "ls"

but azure only off 2 methods (for ContainterClient): 
"listBlobsFlat" which list all files (e.g.all files in your C:/) so this could be very messy and not very useful. and
"listBlobsByHierarchy" will just list top folder. which like you run "dir" under C:/

BlobClient doesn't have above function... which is so bad.

you also can access a file or folder directly by:  containerClient.getBlobClient("2018/11/25/xxx/");

but not able to run anything to list sub files.

https://docs.microsoft.com/en-us/javascript/api/@azure/storage-blob/blockblobclient?view=azure-node-latest#download-undefined---number--undefined---number--blobdownloadoptions-




the only way to get sub files under a folder is:  by using prefix

for await (const blob of containerClient.listBlobsFlat({
      includeMetadata: true,
      prefix: "2018/11/25/xx"
    })) {
    console.log('\t', blob.name);

        }


-------


i have a blob, i need to download files from it.   (i need to download from this folder: 2018/11/25/xxx/)

first i need to list (print) files


const { BlobServiceClient } = require('@azure/storage-blob');       
                                                                                                                           
const CONNECT_STR = "BlobEndpoint=https://xxxxx"

async function listAllBlobs() {                   
const blobServiceClient = await BlobServiceClient.fromConnectionString(CONNECT_STR); 
const containerName = 'yourcontainername';
// Get a reference to a container
const containerClient = await blobServiceClient.getContainerClient(containerName);

// List the blob(s) in the container. all in flat.
// for await (const blob of containerClient.listBlobsFlat()) {
//     console.log('\t', blob.name);
// }       this doesn't work for me, i have a lot of files. 1000000+


for await (const blob of containerClient.listBlobsByHierarchy('/')) {
    console.log('\t', blob.name);
}           
        // above will list all folders, like you run "ls" in mac:
//   2018/    -- this is a folder name
//   2019/   -- this is a folder name

       //if you know the folder, then you can directly access:
var blobx = await containerClient.getBlobClient("2018/11/25/xxx/");       
console.log('\t', blobx.name);           
       //above will just print out the folder name. like you run "ls 2018/11/25/xxx/"
   
      //then you can list the folder from the blob client: (blob client can be a folder, also can be a file)
   
      for await (const blob of blobx.listBlobsFlat()) {
           console.log('\t', blob.name);
      }
     // this will list all files i neeed to download
                         
}                     
                                                                                                                           
listAllBlobs().then(() => console.log('Done')).catch((ex) => console.log(ex.message));






No comments:

Post a comment