Seven cattle third-party resource capture and private resource download

Posted Jun 7, 20203 min read

There is such a demand in the recent project. Because the video recorded by our teacher of Tencent live broadcast software(TClass, Tencent Cloud Interactive Classroom) is stored on Tencent, but Tencent saves the video only for three days, so we need to grab the video to On our Qiuniu cloud, low-frequency storage is required, and storage to private space is required.

Synchronize third-party resources

const qiniu = require('qiniu');
const accessKey ='your accessKey';
const secretKey ='your secretKey';
let mac = new qiniu.auth.digest.Mac(accessKey, secretKey);
let config = new qiniu.conf.Config();
config.zone = qiniu.zone.Zone_z0;

let bucketManager = new qiniu.rs.BucketManager(mac, config);

let resUrl ='https://blog.adityasui.com/_nuxt/img/user.85831cd.jpg';
let bucket ='cris-store';
let key ='sync.png';
bucketManager.fetch(resUrl, bucket, key, function(err, respBody, respInfo) {
  if(err) {
    console.log(err);
  } else {
    if(respInfo.statusCode === 200) {
      console.log(respBody.key);
      console.log(respBody.hash);
      console.log(respBody.fsize);
      console.log(respBody.mimeType);
    } else {
      console.log(respInfo.statusCode);
      console.log(respBody);
    }
  }
});

Third-party resource fetching

Asynchronously grab third-party resources

However, the method of synchronously grabbing third-party resources does not support the inability to pass parameters, so it cannot meet our needs for low-frequency storage, while asynchronously grabbing third-party resources supports passing parameters.

const qiniu = require('qiniu');

const accessKey ='your accessKey';

const secretKey ='your secretKey';

let mac = new qiniu.auth.digest.Mac(accessKey, secretKey);

let bucket ='cris-store';

let resUrl ='https://blog.adityasui.com/_nuxt/img/user.85831cd.jpg';

let key ='async.png';

let fetchUrl = "http://api-z0.qiniu.com/sisyphus/fetch";

const reqBody = {
  url:resUrl,
  bucket,
  file_type:1, //0:standard storage(default), 1:low frequency storage, 2:archive storage
  key,
};

const reqBodyStr = JSON.stringify(reqBody);

qiniu.rpc.post(fetchUrl, reqBodyStr, {
  Authorization:qiniu.util.generateAccessTokenV2(mac, fetchUrl,'POST','application/json', reqBodyStr),
  'Content-Type':'application/json',
},
  function(error, response, body) {
    try {
      if(error) throw error;
      console.log(body);
    } catch(e) {
      console.log(e);
    }
  }

);

Among them, low frequency storage only needs to pass file_type to 1.

Asynchronous third-party resource fetching

I really want to complain about Qi Niu's documentation. Obviously the API has been written, but the API is not written in the documentation.

Private resource download

Compared with the public resource URL, accessing the private resource URL only adds two parameters e and token, which indicate the expiration time and download credentials, because Qi Niu has provided an example of generating download credentials, here is the code directly

const qiniu = require('qiniu');

const accessKey ='your accessKey';
const secretKey ='your secretKey';

let mac = new qiniu.auth.digest.Mac(accessKey, secretKey);

let config = new qiniu.conf.Config();

let bucketManager = new qiniu.rs.BucketManager(mac, config);
let privateBucketDomain ='http://qagyyccko.bkt.clouddn.com';
let deadline = parseInt(Date.now()/1000) + 3600; //1 hour expires Unix timestamp in seconds
let privateDownloadUrl = bucketManager.privateDownloadUrl(privateBucketDomain,'whq.mp4', deadline);

console.log('@privateDownloadUrl', privateDownloadUrl);

But then I encountered another problem. After setting the validity period for 20 seconds, it was found that after 20 seconds, the resources were still accessible. So Google took a long time to find out that the test domain name cannot be used for private space and the custom domain name must be turned on. right. Just configure according to the official website.

Precautions for private spaces
Private Resource Download
Download file

Related Posts