I need to read a large text file of around 5-6 GB line by line using Java.
How can I do this quickly?
2021-6-3 anglehua
I need to read a large text file of around 5-6 GB line by line using Java.
How can I do this quickly?
A common pattern is to use
try (BufferedReader br = new BufferedReader(new FileReader(file))) {
String line;
while ((line = br.readLine()) != null) {
// process the line.
}
}
You can read the data faster if you assume there is no character encoding. e.g. ASCII-7 but it won't make much difference. It is highly likely that what you do with the data will take much longer.
EDIT: A less common pattern to use which avoids the scope of line
leaking.
try(BufferedReader br = new BufferedReader(new FileReader(file))) {
for(String line; (line = br.readLine()) != null; ) {
// process the line.
}
// line is not visible here.
}
UPDATE: In Java 8 you can do
try (Stream<String> stream = Files.lines(Paths.get(fileName))) {
stream.forEach(System.out::println);
}
NOTE: You have to place the Stream in a try-with-resource block to ensure the #close method is called on it, otherwise the underlying file handle is never closed until GC does it much later.
Look at this blog:
The buffer size may be specified, or the default size may be used. The default is large enough for most purposes.
// Open the file
FileInputStream fstream = new FileInputStream("textfile.txt");
BufferedReader br = new BufferedReader(new InputStreamReader(fstream));
String strLine;
//Read File Line By Line
while ((strLine = br.readLine()) != null) {
// Print the content on the console
System.out.println (strLine);
}
//Close the input stream
fstream.close();