How to Remove Duplicates from ArrayList in Java

In this tutorial, we are going to see how to remove duplicates from ArrayList in Java with various inbuild APIs and third-party libraries.

Java List is an interface that facilitates storing an ordered collection of data. It allows storing duplicate values. Many times, we need to avoid duplication in the List. We need to follow a certain methodology to achieve unique list values as out of the box it isn’t available.

With Plain Java API (Remove duplicates from ArrayList in Java)

This is probably the simplest way to remove duplicates from a list as follows.

List<String> nameLst = Arrays.asList("Nilang","Sam","Peter","Denial","Peter");
        
System.out.println("Original List :: "+nameLst);

List<String> uniqueNameLst = new ArrayList<>();
for(String name : nameLst){
    if(!uniqueNameLst.contains(name)){
        uniqueNameLst.add(name);
    }
}
System.out.println("Updated List :: "+uniqueNameLst);

The easiest way to remove duplicates is to create a second list and add only unique values. This may impact performance if the list size is huge. The output is as follows.

Original List :: [Nilang, Sam, Peter, Denial, Peter]
Updated List :: [Nilang, Sam, Peter, Denial]

The elements in the updated list remain in the same order.

With Set in Core Java API (Remove duplicates from a list in Java)

A Set in Java can be used to remove duplicates from a list as follows.

List<String> nameLst = new ArrayList<>();
nameLst.add("Nilang");
nameLst.add("Sam");
nameLst.add("Peter");
nameLst.add("Denial");
nameLst.add("Peter");

System.out.println("Original List :: "+nameLst);
Set<String> uniqueNameSet = new HashSet<>(nameLst);

List<String> uniqueNameLst = new ArrayList<>(uniqueNameSet);
System.out.println("Updated List :: "+uniqueNameLst);

Explanation

We are taking the help of Set to remove duplicates. Set in Java doesn’t allow duplicate. So storing list content into Set first and then create a new list out of it. You can shorten the above code as follows.

List<String> nameLst = Arrays.asList("Nilang","Sam","Peter","Denial","Peter");
        
System.out.println("Original List :: "+nameLst);
Set<String> uniqueNameSet = new HashSet<>(nameLst);
List<String> uniqueNameLst = new ArrayList<>(uniqueNameSet);
System.out.println("Updated List :: "+uniqueNameLst);

The output would be as follows.

Original List :: [Nilang, Sam, Peter, Denial, Peter]
Updated List :: [Peter, Denial, Nilang, Sam]

After using a Set, the order of data in the list is altered. In case if the order of elements is important, you can use LinkedHashSet as follows.

List<String> nameLst = Arrays.asList("Nilang","Sam","Peter","Denial","Peter");
        
System.out.println("Original List :: "+nameLst);
Set<String> uniqueNameSet = new LinkedHashSet<>(nameLst);

List<String> uniqueNameLst = new ArrayList<>(uniqueNameSet);
System.out.println("Updated List :: "+uniqueNameLst);

The LinkedHashSet does not only remove the duplicate elements but maintains the order as well. The output is as follows.

Original List :: [Nilang, Sam, Peter, Denial, Peter]
Updated List :: [Nilang, Sam, Peter, Denial]

With Set in Google Guava API (Remove duplicates from an ArrayList in Java)

Google provides a Guava API, which provides utility classes. They are used to play with the collection framework with fewer lines of code as follows.

List<String> nameLst = Lists.newArrayList("Nilang","Sam","Peter","Denial","Peter");
        
System.out.println("Original List :: "+nameLst);
List<String> uniqueNameLst = new ArrayList<>(Sets.newLinkedHashSet(nameLst));
System.out.println("Updated List :: "+uniqueNameLst);

Explanation

In Guava, Lists is a utility class used to create an ArrayList. Similarly, the Sets class is used to create a new LinkedHashSet. In this example also, we are using the same methodology – add the data first into Set to remove duplicate and then create a new List out of it.

You will get the same output as in the previous case because LinkedHashSet preserves the order of elements.

If you are using the Maven, you can add Guava dependency as follows in your pom.xml.

<dependency>
  <groupId>com.google.guava</groupId>
  <artifactId>guava</artifactId>
  <version>29.0-jre</version>
</dependency>

Using Java 8 Stream.distinct() API (Remove duplicates from a list in Java)

Let’s talk about a new Solution. Since version 8, Java provides Stream API, which you can use to extract the data in different formats. We will use the distinct() method to remove duplicate values. The updated code looks as follows.

List<String> nameLst = Arrays.asList("Nilang","Sam","Peter","Denial","Peter");
    	
System.out.println("Original List :: "+nameLst);
List<String> uniqueNameLst = nameLst.stream()
 .distinct()
 .collect(Collectors.toList());
System.out.println("Updated List :: "+uniqueNameLst);

Explanation

• The stream() method on List provides a Stream of String.
• The distinct() method will return only unique values
• The collect() method uses the Stream of String and populates a List by providing Collectors.toList() parameter.
• The series of method calls (till collect() method) will then return a List with unique values.

The output would be as follows.

Original List :: [Nilang, Sam, Peter, Denial, Peter]
Updated List :: [Nilang, Sam, Peter, Denial]

You will see, Stream API retains the original order of the list while removing the duplicate.

Set Vs Stream API, What to choose, and when?

Most Important question. Which option is the best fit for you? If you are using Java 1.8 or above, go with Stream API. If not then choose LinkedHashSet.

However, there is no hardcode rule. You can still use LinkedHashSet for Java 1.8 or above, but, Stream API is more recommended for a cleaner code and more flexibility.

Summary

• List in Java provides a way to store ordered collections. It also allows duplicate values.
• To remove duplicate from the list, you can use either of the following approaches
  • Create a new List and add only those elements which do not exist.
  • Use LinkedHashSet to remove duplicates and retain the order of the original list. Alternatively use Google Guava API to write shorter code for collection.
  • If you are using Java 1.8 or above, use Stream API. This does not only provide a cleaner code but more flexibility as well.
• In this example, we took an example of a String List. Just follow these approaches and remove other types of List as an exercise. You can also try custom types of List.
• For more detail about ArrayList, head over to this link.
• We have seen how good ArrayList is for working with various collections of data. Java collection framework provides many more collections that you can use in day-to-day programming. LinkedList is one of them. Check out the next article on LinkedList over here.