Java SE 8: Creating an HTTP Link Checker with Java

In this section, you use the Java SE 8 regular expression to validate the URL format. The HTTPClient.java file has a verifyUrl method, which accepts the URL as the input parameter.

1. Import the following packages:

import java.net.HttpURLConnection;
import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

2. Add the following method to the HTTPClient.java file to verify the URL format:

public class HTTPClient {    
        private boolean verifyUrl(String url) {
        String urlRegex = "^(http|https)://[-a-zA-Z0-9+&@#/%?=~_|,!:.;]*[-a-zA-Z0-9+@#/%=&_|]";
        Pattern pattern = Pattern.compile(urlRegex);
        Matcher m = pattern.matcher(url);
        if (m.matches()) {
            return true;
        } else {
            return false;
         }
    }



The verifyUrl method verifies the url parameter passed as the input parameter by matching it with the regular expression. If the match is successful, then it returns true; otherwise, it returns false.

3. Review the code. Your code should look like the following:

    
   

In this section, you validate the URLs listed in the url-list.txt file.

1. Modify HTTPClient.java.

   1. Declare the following variables:
      

      private String failedURLS ="";
private String succeededURLS ="";
private String incorrectURLS = "";

   2. Retrieve the URLs from the url-list.txt file.

      public void validateUrl() throws Exception {
    Path filePath = Paths.get("src/url-list.txt");
    List<String> myURLArrayList = Files.readAllLines(filePath); 


      You retrieve the file location by using the get method with the Paths class. Using the readAllLines method, you read the URLs in the filePath into the List of type string.
   
   
   3. Invoke the verifyUrl method.

      myURLArrayList.forEach((String url) -> {
        if (verifyUrl(url)) {
            try{

      Here, you're using the For-Each loop, which you write with a lambda expression. The For-Each loop retrieves the URL from myURLArrayList, and the retrieved URL is passed as an input parameter to the verifyUrl method. The verifyUrl method returns true for a valid URL format; otherwise, it returns false. If the verifyUrl method returns true, then the if condition is true, thereby executing the code in the try block.
      

   4. Create the HttpURLConnection connection.

      URL myURL = new URL(url);
HttpURLConnection myConnection = (HttpURLConnection) myURL.openConnection();


      You will open the myURL instance with the connection that you created in this step.

   5. Validate the URL with the response code.

      if (myConnection.getResponseCode()==URLStatus.HTTP_OK.getStatusCode()) {
            succeededURLS = succeededURLS + "\n" + url + "****** Status message is : "
                + URLStatus.getStatusMessageForStatusCode(myConnection.getResponseCode());
 } else {
       failedURLS = failedURLS + "\n" + url + "****** Status message is : "
           + URLStatus.getStatusMessageForStatusCode(myConnection.getResponseCode());
 }


      The myConnection instance receives the URL's response code and verifies the status. If the status code is 200 (HTTP_OK), then the URL is classified as succeededURLS,  otherwise, it's classified as failedURLS.

   6. Close try with the catch block.

      } catch (Exception e) {
           System.out.print("For url- " + url+ "" +e.getMessage());
  }

      The catch block is executed when an exception is thrown when  HttpURLConnection is created and opened.

   7. Verify the incorrect URLs.

        }else {
        incorrectURLS += "\n" + url;
      }
  });
}

      The else block is executed when the verifyUrl method returns false because the URL validation failed.

2. Review the code. Your code should look like the following:

    
   

3. Add the following code to the main() method in the HTTPClient.java file:

   public static void main(String[] args) {

  try {
     HTTPClient myClient = new HTTPClient();
     myClient.validateUrl();
      System.out.println("Valid URLS that have successfully connected :");
      System.out.println(myClient.succeededURLS);
      System.out.println("\n--------------\n\n");
      System.out.println("Broken URLS that did not successfully connect :");
      System.out.println(myClient.failedURLS);
    } catch (Exception e) {
        System.out.print(e.getMessage()); }
      }
  }

   The main()method creates an instance of HTTPClient named myClient. Using the myClient instance, you invoke the validateUrl method. The myClient instance displays the valid URLs that connected, the broken URLs that did not connect, and the status code in the console.

4. Review the code. Your code should look like the following:

    
   

5. On the Projects tab, right-click HTTPClient.java and select Run File.



6. Review the set of URLs displayed in the url-list.txt file. 



For the given set of URLs, the application retrieves each URL, verifies it, validates it, and classifies it accordingly.

7. Verify the output.



You successfully used the URLs listed in the url-list.txt file and classified them as valid URLs or broken URLs. The status codes of the broken URLs are displayed in the console.