Multi tool use

Set with duplicates java - import from file - java

I have a small project.

The project imports the txt file to String (coding similar to CSV - contains semicolons = ";").

In the next steps, the String is changed to ArrayList.

Then, using Predicate, I remove elements that do not interest me.

At the end I replace ArrayList on TreeSet to remove duplicates.
Unfortunately, there is a problem here because the duplicates occur ...

I checked in Notepadd ++ changing the encoding on ANSI to check whether there are no unnecessary characters.

Unfortunately, everything looks good and duplicates are still there.

Uploaded input file - https://drive.google.com/open?id=1OqIKUTvMwK3FPzNvutLu-GYpvocUsSgu

Any idea?

public class OpenSCV {
private static final String SAMPLE_CSV_FILE_PATH = "/Downloads/all.txt";

public static void main(String args) throws IOException {

File file = new File(SAMPLE_CSV_FILE_PATH);
String str = FileUtils.readFileToString(file, "utf-8");
str = str.trim();
String str2 = str.replace("n", ";").replace(""", "" ).replace("nn",";").replace("*www.*","")
.replace("u0000","").replace(",",";").replace(" ","").replaceAll(";{2,}",";");

List<String> lista1 = new ArrayList<>(Arrays.asList((str2.split(";"))));

Predicate<String> predicate = s -> !(s.contains("@"));

Set<String> removeDuplicates = new TreeSet<>(lista1);

removeDuplicates.removeIf(predicate);

String fileName2 = "/Downloads/allMails.txt";
try ( BufferedWriter bw =
new BufferedWriter (new FileWriter (fileName2)) )
{
for (String line : removeDuplicates) {
bw.write (line + "n");
}
bw.close ();
} catch (IOException e) {
e.printStackTrace ();
}
}
}


Set<String>

.replaceAll(";;",";").replaceAll(";;;",";").replaceAll(";;;;",";").replaceAll(";;;;;",";");

replaceAll(";{2,}",";")

bw.close ()

try(BufferedWriter bw = …)

replace

replaceAll

1 Answer
1

before doing str.replace you can try str.trim to remove any spaces or unwanted and unseen characters.

str = str.trim()


By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.