How to Remove Duplicates from a List in Java

In this tutorial, we are going to see how to remove duplicates from a List in Java with various inbuild APIs and third-party libraries.

Remove duplicate from a list in Java

Java List is an interface that facilitates storing an ordered collection of data. It allows storing duplicate values. Many times, we need to avoid duplication in the List. We need to follow a certain methodology to achieve unique list values as out of the box it isn’t available.

With Plain Java API (Remove duplicates from a list in Java)

This is the simplest way to remove duplicates from a list as follows.

List<String> nameLst = Arrays.asList("Nilang","Sam","Peter","Denial","Peter");
        
System.out.println("Original List :: "+nameLst);

List<String> uniqueNameLst = new ArrayList<>();
for(String name : nameLst){
    if(!uniqueNameLst.contains(name)){
        uniqueNameLst.add(name);
    }
}
System.out.println("Updated List :: "+uniqueNameLst);

The easiest way to remove duplicate is to create a second list and add only those elements which do not exist already. This may impact performance if the list size is huge. The output is as follows.

Original List :: [Nilang, Sam, Peter, Denial, Peter]
Updated List :: [Nilang, Sam, Peter, Denial]

The elements in the updated list remain in the same order.

With Set in Core Java API (Remove duplicates from a list in Java)

A Set in Java can be used to remove duplicates from a list as follows.

List<String> nameLst = new ArrayList<>();
nameLst.add("Nilang");
nameLst.add("Sam");
nameLst.add("Peter");
nameLst.add("Denial");
nameLst.add("Peter");

System.out.println("Original List :: "+nameLst);
Set<String> uniqueNameSet = new HashSet<>(nameLst);

List<String> uniqueNameLst = new ArrayList<>(uniqueNameSet);
System.out.println("Updated List :: "+uniqueNameLst);

Explanation

We are taking the help of Set to remove duplicates. Set in Java doesn’t allow duplicate. So storing list content into Set first and then create a new list out of it. You can shorten the above code as follows.

List<String> nameLst = Arrays.asList("Nilang","Sam","Peter","Denial","Peter");
        
System.out.println("Original List :: "+nameLst);
Set<String> uniqueNameSet = new HashSet<>(nameLst);
List<String> uniqueNameLst = new ArrayList<>(uniqueNameSet);
System.out.println("Updated List :: "+uniqueNameLst);

The output would be as follows.

Original List :: [Nilang, Sam, Peter, Denial, Peter]
Updated List :: [Peter, Denial, Nilang, Sam]

After using a Set, the order of data in the list is altered. In case if the order of elements is important, you can use LinkedHashSet as follows.

List<String> nameLst = Arrays.asList("Nilang","Sam","Peter","Denial","Peter");
        
System.out.println("Original List :: "+nameLst);
Set<String> uniqueNameSet = new LinkedHashSet<>(nameLst);

List<String> uniqueNameLst = new ArrayList<>(uniqueNameSet);
System.out.println("Updated List :: "+uniqueNameLst);

The LinkedHashSet does not only remove the duplicate elements but maintain the order as well. The output is as follows.

Original List :: [Nilang, Sam, Peter, Denial, Peter]
Updated List :: [Nilang, Sam, Peter, Denial]

With Set in Google Guava API (Remove duplicates from a list in Java)

Google provides a Guava API, which provides utility classes that are used to play with the collection framework with a fewer line of code as follows.

List<String> nameLst = Lists.newArrayList("Nilang","Sam","Peter","Denial","Peter");
        
System.out.println("Original List :: "+nameLst);
List<String> uniqueNameLst = new ArrayList<>(Sets.newLinkedHashSet(nameLst));
System.out.println("Updated List :: "+uniqueNameLst);

Explanation

In Guava, Lists is a utility class used to create an ArrayList. Similarly, the Sets class is used to create a new LinkedHashSet. In this example also, we are using the same methodology – add the data first into Set to remove duplicate and then create a new List out of it.

You will get the same output as in the previous case. Since we are using LinkedHashSet, the order of elements is preserved.

If you are using the Maven, you can add Guava dependency as follows in your pom.xml.

<dependency>
  <groupId>com.google.guava</groupId>
  <artifactId>guava</artifactId>
  <version>29.0-jre</version>
</dependency>

Using Java 8 Stream.distinct() API (Remove duplicates from a list in Java)

Let’s talk about a new Solution. Since version 8, Java provides Stream API which is used to extract the data in different formats. We will use the distinct() method to remove duplicate values. The updated code looks as follows.

List<String> nameLst = Arrays.asList("Nilang","Sam","Peter","Denial","Peter");
    	
System.out.println("Original List :: "+nameLst);
List<String> uniqueNameLst = nameLst.stream()
 .distinct()
 .collect(Collectors.toList());
System.out.println("Updated List :: "+uniqueNameLst);

Explanation

  • The stream() method on List provides a Stream of String.
  • The distinct() method will return only unique values
  • The collect() method uses the Stream of String and populates a List by providing Collectors.toList() parameter.
  • The series of method calls (till collect() method) will then return a List with unique values.

The output would be as follows.

Original List :: [Nilang, Sam, Peter, Denial, Peter]
Updated List :: [Nilang, Sam, Peter, Denial]

You will see, Stream API retains the original order of the list while removing the duplicate.

Set Vs Stream API, What to choose, and when?

Most Important question. Which option is the best fit for you? If you are using Java 1.8 or above, go with Stream API. If not then choose LinkedHashSet.

However, there is no hardcode rule. You can still use LinkedHashSet for Java 1.8 or above, but still, Stream API is more recommended for a cleaner code and more flexibility.

Summary

  • List in Java provides a way to store ordered collections. It also allows duplicate values.
  • To remove duplicate from the list, you can use either of the following approaches
    • Create a new List and add only those elements which do not exist.
    • Use LinkedHashSet to remove duplicates and retain the order of the original list.¬†Alternatively use Google Guava API to write shorter code for collection.
    • If you are using Java 1.8 or above, use Stream API. This does not only provide cleaner code but more flexibility as well.
  • In this example, we took an example of a String List. Just follow these approaches and remove other types of List as an exercise. You can also try custom types of List.

Recommended For You

About the Author: Nilang

Nilang Patel is a technology evangelist who loves to spread knowledge and helping people in all possible ways. He is an author of two technical books - Java 9 Dependency and Spring 5.0 Projects.

4 Comments

    1. Hello Dhaval,
      Thanks for the appreciation and welcome to the Tech blog. Feel free to give a suggestion or specific topics you want to get an article.

      Regards
      Nilang

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.