网络图片的下载以及上传到fastDFS

最近做了一次下载网络图片然后上传到fastDFS的任务。碰到了个别小问题现在记录一下。

主要思路

1. 下载图片，然后，生成临时文件
2. 得到临时文件生成的文件流
3. 上传该文件流到fastDFS。

系统分析

1. 网络图片下载

  public static File downloadFromUrl(String urlStr){ //获取URL对象 URL url = null; HttpURLConnection conn = null; File file = null; try { url = new URL(urlStr); //获取连接 conn = (HttpURLConnection) url.openConnection(); conn.setConnectTimeout(6000); //设置超时时间是3秒 conn.setReadTimeout(6000); //防止屏蔽程序抓取而返回403错误 conn.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.19 Safari/537.36"); //得到临时文件 InputStream is = conn.getInputStream(); if (is == null || is.available() <= 0) { throw new DownloadException("图片为空"); } file = getTemplateFile(is); } catch (MalformedURLException e) { log.error("图片下载出现异常={}", e.getMessage()); throw new DownloadException("图片下载失败"); } catch (Exception e) { log.error("图片下载出现异常={}", e.getMessage()); throw new DownloadException("图片下载失败"); } finally { //关闭连接 if (conn != null) { conn.disconnect(); } } return file; }
  

 

  
1
  
2
  
3
  
4
  
5
  
6
  
7
  
8
  
9
  
10
  
11
  
12
  
13
  
14
  
15
  
16
  
17
  
18
  
19
  
20
  
21
  
22
  
23
  
24
  
25
  
26
  
27
  
28
  
29
  
30
  
31
  
32
  
33
  
34
 

此处得到一个输入流之后生成了一个临时文件，至于为啥要生成临时文件，主要的原因是如果直接将下载得到的输入流上传到fastDFS 上会导致上传之后的图片显示不全。故，做了如下如下处理。
生成临时文件：

  /** * 获取模板文件--获取到的文件为临时文件，用完后需要手动删除 * * @param inputStream * @return 模板文件 * @throws Exception 异常抛出 */ public static File getTemplateFile(InputStream inputStream) throws Exception { File file = File.createTempFile("temp_image", null); inputStreamToFile(inputStream, file); if (file.exists() && file.length() <= 0) { throw new DownloadException("临时文件为空"); } return file; }

/** * InputStream 转file * * @param ins  输入流 * @param file 目标文件 */ public static void inputStreamToFile(InputStream ins, File file) { try { OutputStream os = new FileOutputStream(file); try { int bytesRead = 0; byte[] buffer = new byte[8192]; while ((bytesRead = ins.read(buffer, 0, 8192)) != -1) { os.write(buffer, 0, bytesRead); } os.flush(); } finally { if (os != null) { os.close(); } if (ins != null) { ins.close(); } } } catch (Exception e) { e.printStackTrace(); } }

  

 

  
1
  
2
  
3
  
4
  
5
  
6
  
7
  
8
  
9
  
10
  
11
  
12
  
13
  
14
  
15
  
16
  
17
  
18
  
19
  
20
  
21
  
22
  
23
  
24
  
25
  
26
  
27
  
28
  
29
  
30
  
31
  
32
  
33
  
34
  
35
  
36
  
37
  
38
  
39
  
40
  
41
  
42
  
43
  
44
  
45
 

1. 图片上传

 @Override public String download2UploadImg(String oldUrl) throws Exception { //1.下载图片 File tempFile = null; InputStream is = null; String newPic; try { tempFile = ImageUtil.downloadFromUrl(oldUrl); String fileName; if (oldUrl.contains(SPECIAL_SIGN)) { fileName = oldUrl.substring(oldUrl.lastIndexOf("/") + 1, oldUrl.indexOf(SPECIAL_SIGN)); } else { fileName = oldUrl.substring(oldUrl.lastIndexOf("/") + 1); } //校验格式 String suffix = fileName.substring(fileName.indexOf(".")); if (!ImageContentType.toMap().containsKey(suffix)) { log.info("文件格式不对，该格式为={}", suffix); throw new ImageException("图片格式不对"); } is = new FileInputStream(tempFile); if (tempFile == null || is == null || is.available() <= 0) { log.info("图片为空"); throw new ImageException("图片为空"); } //2.上传图片 newPic = FastDfsUtil.uploadFile(is, fileName); } finally { //手动删除临时文件 if (is != null) { try { is.close(); } catch (IOException e) { throw new IOException("文件流关闭失败"); } } if (tempFile != null) { tempFile.delete(); } } log.info("上传完之后新图片地址="+(fastdfs_url + newPic)); return fastdfs_url + newPic; }
  

 

  
1
  
2
  
3
  
4
  
5
  
6
  
7
  
8
  
9
  
10
  
11
  
12
  
13
  
14
  
15
  
16
  
17
  
18
  
19
  
20
  
21
  
22
  
23
  
24
  
25
  
26
  
27
  
28
  
29
  
30
  
31
  
32
  
33
  
34
  
35
  
36
  
37
  
38
  
39
  
40
  
41
  
42
  
43
  
44
 

具体的上传方式参见：上传图片到fastDFS
3. 测试方法

  @Test public void testDownload2Upload() throws Exception { //1.图片正常下载上传 String oldUrl = "http://img.okwuyou.com/shop/store/goods/1/2017/05/16/1_05482671770179845.jpg"; Assert.assertNotNull(okwuyouSynItemService.download2UploadImg(oldUrl)); //2.图片为空 String oldUrl1 = "http://img.okwuyou.com/shop/store/goods/1/2018/01/25/48_05701903155753607.jpg"; try { okwuyouSynItemService.download2UploadImg(oldUrl1); } catch (Exception e) { Assert.assertTrue(e instanceof DownloadException); } //3.图片路径失效 String oldUrl2 = "http://img.okwuyou.com/shop/store/goods/12017/12/15/25_05666488135210549.png"; try { okwuyouSynItemService.download2UploadImg(oldUrl2); } catch (Exception e) { Assert.assertTrue(e instanceof DownloadException); } //4.图片格式不对 String oldUrl3 = "https://github.com/XWxiaowei/JavaWeb/blob/master/jackson-demo/pom.xml"; try { okwuyouSynItemService.download2UploadImg(oldUrl3); } catch (Exception e) { Assert.assertTrue(e instanceof ImageException); } } @Test public void testDeleteFile() { //5.临时文件是否删除
// 1.获取临时文件的地址 String tempPath = System.getProperty("java.io.tmpdir") + File.separator; System.out.println("临时文件的目录是=" + tempPath);
// 2.匹配临时文件 List<String> fileNameList = getAllFile(tempPath, false); if (!CollectionUtils.isEmpty(fileNameList)) { int i = 0; for (String s : fileNameList) { if (s.contains("temp_image")) { i++; System.out.println("临时文件未删除" + i); } } } } /** * 获取路径下的所有文件/文件夹 * @param directoryPath 需要遍历的文件夹路径 * @param isAddDirectory 是否将子文件夹的路径也添加到list集合中 * @return */ public static List<String> getAllFile(String directoryPath,boolean isAddDirectory) { List<String> list = new ArrayList<String>(); File baseFile = new File(directoryPath); if (baseFile.isFile() || !baseFile.exists()) { return list; } File[] files = baseFile.listFiles(); for (File file : files) { if (file.isDirectory()) { if(isAddDirectory){ list.add(file.getAbsolutePath()); } list.addAll(getAllFile(file.getAbsolutePath(),isAddDirectory)); } else { list.add(file.getAbsolutePath()); } } return list; }
  

 

  
1
  
2
  
3
  
4
  
5
  
6
  
7
  
8
  
9
  
10
  
11
  
12
  
13
  
14
  
15
  
16
  
17
  
18
  
19
  
20
  
21
  
22
  
23
  
24
  
25
  
26
  
27
  
28
  
29
  
30
  
31
  
32
  
33
  
34
  
35
  
36
  
37
  
38
  
39
  
40
  
41
  
42
  
43
  
44
  
45
  
46
  
47
  
48
  
49
  
50
  
51
  
52
  
53
  
54
  
55
  
56
  
57
  
58
  
59
  
60
  
61
  
62
  
63
  
64
  
65
  
66
  
67
  
68
  
69
  
70
  
71
 

文章来源: feige.blog.csdn.net，作者：码农飞哥，版权归原作者所有，如需转载，请联系作者。

原文链接：feige.blog.csdn.net/article/details/80910968