Building a Spring Boot Web Application with Bootstrap for Scraping Electric Car Data from EV Database

13 Feb, 2024

Introduction:

In this tutorial, we'll build a Spring Boot web application that scrapes electric car data from the EV Database using Jsoup. We'll enhance the appearance of the web interface by integrating Bootstrap, a popular front-end framework. By the end of this tutorial, you'll have a sleek and functional web application to display scraped electric car data.

ev image

Prerequisites:

Basic understanding of Java and Spring Boot
Maven installed on your machine
IDE (such as IntelliJ IDEA or Eclipse) installed

Step 1: Create a Spring Boot Project

Create a new Spring Boot project using Spring Initializr or your preferred IDE. Add the necessary dependencies, including Spring Web, Jsoup, and Thymeleaf, to your project's pom.xml file.

Step 2: Create a Controller Class¹

Create a new Java class to act as a controller for handling HTTP requests. Define a method to scrape electric car data from the EV Database using Jsoup and return it as a JSON response.

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

@RestController
public class CarController {

    @GetMapping("/scrape-cars")
    public List<Car> scrapeCars() {
        List<Car> cars = new ArrayList<>();
        try {
            // URL of the page you want to scrape
            String url = "https://ev-database.org/";

            // Send a GET request to the URL and parse the HTML
            Document document = Jsoup.connect(url).get();

            // Find the specific data you're interested in
            // For example, let's say you want to extract the names and prices of electric cars within a price range
            Elements carElements = document.select("div.car-wrapper");

            // Extract the names and prices of the electric cars
            for (Element carElement : carElements) {
                Element titleElement = carElement.selectFirst("div.car-title");
                String title = titleElement.text();

                Element priceElement = carElement.selectFirst("div.price");
                String priceText = priceElement.text();
                int price = Integer.parseInt(priceText.replaceAll("[^0-9]", ""));

                // Define your price range here
                int minPrice = 30000;
                int maxPrice = 60000;

                // Check if the car's price is within the specified range
                if (price >= minPrice && price <= maxPrice) {
                    cars.add(new Car(title, priceText));
                }
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
        return cars;
    }
}

Step 3: Create a Model Class²

Define a simple model class to represent the scraped car data.

public class Car {
    private String name;
    private String price;

    public Car(String name, String price) {
        this.name = name;
        this.price = price;
    }

    // Getters and setters
}

Step 4: Display Data in a Web Interface with Bootstrap

Create a basic HTML template to display the scraped car data. Integrate Bootstrap to enhance the appearance and layout of the web interface.

<!DOCTYPE html>
<html lang="en" xmlns:th="http://www.thymeleaf.org">
<head>
    <meta charset="UTF-8">
    <title>Scraped Cars</title>
    <!-- Bootstrap CSS -->
    <link href="https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css" rel="stylesheet">
</head>
<body>
    <div class="container">
        <h1 class="mt-4 mb-4">Electric Cars</h1>
        <ul class="list-group">
            <li th:each="car : ${cars}" class="list-group-item">
                <span th:text="${car.name}"></span> - <span th:text="${car.price}"></span>
            </li>
        </ul>
    </div>
    <!-- Bootstrap JS (optional, for certain components) -->
    <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/js/bootstrap.min.js"></script>
</body>
</html>

Step 5: Run the Application

Start the Spring Boot application, and navigate to http://localhost:8080/scrape-cars in your web browser to view the scraped electric car data with the enhanced appearance provided by Bootstrap.

Conclusion:

In this tutorial, we've demonstrated how to build a Spring Boot web application to scrape electric car data from EV Database using Jsoup. By integrating Bootstrap, we've enhanced the appearance and layout of the web interface, resulting in a sleek and functional application for displaying scraped data. Happy coding!

A controller is a Java class responsible for handling incoming HTTP requests and providing appropriate responses. In the context of a Spring Boot application, controllers are annotated with @Controller or @RestController annotations to indicate that they are responsible for handling web requests. In summary, the controller class CarController defines an endpoint ("/scrape-cars") that clients can access to retrieve scraped electric car data. When a client sends a GET request to this endpoint, the scrapeCars() method is executed, and the scraped data is returned as a JSON response.↩
A model class represents the data structure used within the application. It typically contains fields that correspond to the attributes of a particular entity or object. In this tutorial, the model class Car represents the structure of an electric car, including its name and price. In summary, the Car model class defines the structure of an electric car by specifying its attributes (name and price). Instances of the Car class are used to represent individual cars within the Spring Boot application, such as when scraping data from EV Database and returning it to the client.↩