FindDuplicateFileInSystem [source code]
public class FindDuplicateFileInSystem {
static
/******************************************************************************/
public class Solution {
@SuppressWarnings("unchecked")
public List<List<String>> findDuplicate(String[] paths) {
Map<String, List<String>> map = new HashMap<>();
for (String s : paths) {
String[] level1 = s.split(" ");
int len = level1.length;
String path = level1[0];
for (int i = 1; i < len; i++) {
String[] level2 = level1[i].split("\\(");
String name = level2[0], content = level2[1].split("\\)")[0];
List<String> ls = map.get(content);
String info = path + "/" + name;
if (ls == null) {
ls = new ArrayList<String>();
ls.add(info);
map.put(content, ls);
} else {
ls.add(info);
}
}
}
List<List<String>> res = new ArrayList<>();
for (Map.Entry e : map.entrySet()) {
List<String> ls = (ArrayList<String>) e.getValue();
if (ls.size() > 1)
res.add(ls);
}
return res;
}
}
/******************************************************************************/
public static void main(String[] args) {
FindDuplicateFileInSystem.Solution tester = new FindDuplicateFileInSystem.Solution();
}
}
题目本身属于比较简单的, 一个反向建表就完成了; 最后的速度是83ms (58%);
Follow-Up的问题都有点懵, 直接看答案了;
找所有duplicate的问题, 好像是不是做过? 如果没有做过我是怎么想到直接反向建表的方法的? 突然自己也有点摸不清了;
这个是editorial评论里面提到的一个优化:
we may not need to remove ( and ) around the content as they can be treated as part of the key too(key and (key) are the same in this algorithm):
这个确实是, 没有想到的; 不过可能是因为case太大了, 最后加了这个优化, submit之后速度并没有一个明显的提升;
discussion里面给出的Follow-Up的答案:
Follow up questions:
- Imagine you are given a real file system, how will you search files? DFS or BFS ?
In general, BFS will use more memory then DFS. However BFS can take advantage of the locality of files in inside directories, and therefore will probably be faster
这个题目的答案大概意思就是, 你自己可以traverse整个文件系统, 而PATH也可以理解为你走到一个位置你就有一个API什么的能够让你自动获取; 这个时候一个好处就在于, 你在同一个directory里面处理完了之后, directory PATH这一块儿的变动比较小, 所以不需要频繁的string操作; 而且如果你先处理完了一个directory里面的所有文件, 那么在这里其实你只需要一次API call(不过同directory的处理好像跟这个问题没有什么关系, 就算是DFS应该也是这个做法);
- If the file content is very large (GB level), how will you modify your solution?
In a real life solution we will not hash the entire file content, since it's not practical. Instead we will first map all the files according to size. Files with different sizes are guaranteed to be different. We will than hash a small part of the files with equal sizes (using MD5 for example). Only if the md5 is the same, we will compare the files byte by byte
也是一个妥协的做法. size这个我是想到了的; 这种关注于practical的Follow-Up的答案好像一般都不是特别复杂, 不要把自己绕进去就行, 提出一个可行可实施的方案就Okay;
If you can only read the file by 1kb each time, how will you modify your solution?
This won't change the solution. We can create the hash from the 1kb chunks, and then read the entire file if a full byte by byte comparison is required.What is the time complexity of your modified solution? What is the most time consuming part and memory consuming part of it? How to optimize?
Time complexity is O(n^2 * k) since in worse case we might need to compare every file to all others. k is the file sizeHow to make sure the duplicated files you find are not false positive?
We will use several filters to compare: File size, Hash and byte by byte comparisons.
这个问题我刚开始还真没有想到怎么回答, 因为感觉这个东西是trivially obvious的. 不过真正要是面试的时候摊上了这样的问题, 还是应该dumb down, 然后用最轻声细语的方式来说就行了; 说够了面试官自然会让你停的;
这个使用了一点JAVA8的解:
public static List<List<String>> findDuplicate(String[] paths) {
Map<String, List<String>> map = new HashMap<>();
for(String path : paths) {
String[] tokens = path.split(" ");
for(int i = 1; i < tokens.length; i++) {
String file = tokens[i].substring(0, tokens[i].indexOf('('));
String content = tokens[i].substring(tokens[i].indexOf('(') + 1, tokens[i].indexOf(')'));
map.putIfAbsent(content, new ArrayList<>());
map.get(content).add(tokens[0] + "/" + file);
}
}
return map.values().stream().filter(e -> e.size() > 1).collect(Collectors.toList());
}
大概看看就可以, 他这个indexOf的使用还是值得学习的; 虽然括号其实是可以不管的;
不过他这里又不是split, 而是用的indexOf, 所以他这里是肯定要管的;
这个是另外一个解:
public class Solution {
public List<List<String>> findDuplicate(String[] paths) {
List<List<String>> result = new ArrayList<List<String>>();
int n = paths.length;
if (n == 0) return result;
Map<String, Set<String>> map = new HashMap<>();
for (String path : paths) {
String[] strs = path.split("\\s+");
for (int i = 1; i < strs.length; i++) {
int idx = strs[i].indexOf("(");
String content = strs[i].substring(idx);
String filename = strs[0] + "/" + strs[i].substring(0, idx);
Set<String> filenames = map.getOrDefault(content, new HashSet<String>());
filenames.add(filename);
map.put(content, filenames);
}
}
for (String key : map.keySet()) {
if (map.get(key).size() > 1) {
result.add(new ArrayList<String>(map.get(key)));
}
}
return result;
}
}
大概思路是一样的, 他这里用了Set而不是List作为Map的value;
注意很多discussion的最后姐最后对Map的这个pass的处理其实是跟他这里一样的, iterate的是key而不是entry; 因为key的type是固定的, 所以直接iterate key好像不需要考虑cast什么的;
或许应该是这样的原因, 他们是对key进行get, 这个出来的type我们直接知道, 因为这个是declare的时候就说了的, 就是List; 但是我的是一个类似对Entry进行get的操作, 这个可能默认的出来的就是object, 也没有办法改变的;
如果以后还碰到这种问题, 想一想这个小转弯. 如果面试的时候出现这个问题, 多半面试官是指望你想到iterate key这个思路的;
总体这个题目不难, 看看差不多了, Follow-Up问题值得好好掌握;
Problem Description
Given a list of directory info including directory path, and all the files with contents in this directory, you need to find out all the groups of duplicate files in the file system in terms of their paths.
A group of duplicate files consists of at least two files that have exactly the same content.
A single directory info string in the input list has the following format:
"root/d1/d2/.../dm f1.txt(f1_content) f2.txt(f2_content) ... fn.txt(fn_content)"
It means there are n files (f1.txt, f2.txt ... fn.txt with content f1_content, f2_content ... fn_content, respectively) in directory root/d1/d2/.../dm. Note that n >= 1 and m >= 0. If m = 0, it means the directory is just the root directory.
The output is a list of group of duplicate file paths. For each group, it contains all the file paths of the files that have the same content. A file path is a string that has the following format:
"directory_path/file_name.txt"
Example 1:
Input:
["root/a 1.txt(abcd) 2.txt(efgh)", "root/c 3.txt(abcd)", "root/c/d 4.txt(efgh)", "root 4.txt(efgh)"]
Output:
[["root/a/2.txt","root/c/d/4.txt","root/4.txt"],["root/a/1.txt","root/c/3.txt"]]
Note:
- No order is required for the final output.
- You may assume the directory name, file name and file content only has letters and digits, and the length of file content is in the range of [1,50].
- The number of files given is in the range of [1,20000].
- You may assume no files or directories share the same name in the same directory.
- You may assume each given directory info represents a unique directory. Directory path and file info are separated by a single blank space.
Follow-up beyond contest: - Imagine you are given a real file system, how will you search files? DFS or BFS?
- If the file content is very large (GB level), how will you modify your solution?
- If you can only read the file by 1kb each time, how will you modify your solution?
- What is the time complexity of your modified solution? What is the most time-consuming part and memory consuming part of it? How to optimize?
- How to make sure the duplicated files you find are not false positive?
Difficulty:Medium
Total Accepted:4.8K
Total Submissions:9K
Contributor: fallcreek
Companies
dropbox
Related Topics
hash table string