I am using axios to scrape some data from some websites. Currently, I hit the case where the encoding of the website is in windows-1251 and the response I got from the request wasn’t readable at all.
Here is a simple way to fix it:
axiosResponseTransformer: AxiosTransformer = (data: any, headers?: any) => { const iconv = new Iconv("windows-1251", "utf-8"); data = iconv.convert(data); return data }; axiosConfig: AxiosRequestConfig = { responseType: "arraybuffer", transformResponse: this.axiosResponseTransformer }; client = axios.create(this.axiosConfig);
Just install the iconv package and then register a transformer that will convert the data from binary (“arraybuffer”) to data with utf-8 encoding. đŸ™‚