laravel 11 import large csv file into database

laravel 11 import large csv file into database guide you how to import a huge number of records CSV file in Laravel application using database seeder.

If we have thousands or millions of records CSV files and we want to store the records in our database, in most cases we use the cron or queue jobs. But we can use the database seeder to import the records if we only want to import the CSV file once a time.

When importing large amounts of data using the import functionality, it often triggers maximum execution time errors due to the file size overwhelming the application’s capacity to handle multiple requests. To address this, we’ve included a prime example of code that efficiently inserts numerous lines from a CSV file into the database within minutes or even seconds.

How to Import Large CSV file in Laravel

Here are the following topics we will implement to make how to import large CSV file in Laravel 11.

#1 Install Laravel Application

First, you need to create a fresh Laravel application by adding the following command in the terminal.

composer create-project --prefer-dist laravel/laravel laravel-import-csv

Next, Go into the app directory:

cd laravel-import-csv

#2 Setup Database Credentials

Now, open the .env file and add your database credentials such as database name, username and password just like below.

DB_CONNECTION=mysql
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=#db_name
DB_USERNAME=#db_username
DB_PASSWORD=#db_password

#3 Generate Model and Migration

Next, you need to execute the following command on the terminal to generate the model, and migration file.

php artisan make:model Community -m

Now open database/migrations/create_communities_table.php file and update the following code on it.

public function up()
{
    Schema::create('communities', function (Blueprint $table) {
        $table->id();
        $table->string('state');
        $table->string('community');
        $table->string('district');
        $table->timestamps();
    });
}

Next migrate the migration to generate the tables in the database.

php artisan migrate

Next, open the app/Models/Community.php file and update the fillable properties on it.

<?php

namespace App\Models;

use Illuminate\Database\Eloquent\Factories\HasFactory;
use Illuminate\Database\Eloquent\Model;

class Community extends Model
{
    use HasFactory;
    protected $guarded = [];
}

#4 Create a Seeder

Now, for importing a large number of CSV in Laravel we will use a seeder for it. So you just need to create a seeder file using the following command.

php artisan make:seeder CommunitySeeder

#5 Update Import Large CSV Code in Seeder

Next, open database\seeders\CommunitySeeder.php file and put the below code on it.

<?php

namespace Database\Seeders;

  use Illuminate\Database\Seeder;
  use App\Models\LocalCommunity;
  use Illuminate\Support\Facades\DB;
  use Illuminate\Support\LazyCollection;

  class CommunitySeeder extends Seeder
  {
    /**
     * Run the database seeds.
    *
    * @return void
    */
    public function run()
    {
        DB::disableQueryLog();
        DB::table('communities')->truncate();
        
        LazyCollection::make(function () {
        $handle = fopen(public_path("comunities.csv"), 'r');

        while (($line = fgetcsv($handle, 4096)) !== false) {
            $dataString = implode(", ", $line);
            $row = explode(';', $dataString);
            yield $row;
        }
        fclose($handle);
        })
        ->skip(1)
        ->chunk(1000)
        ->each(function (LazyCollection $chunk) {

            $records = $chunk->map(function ($row) {
            return [
                "state" => $row[0],
                "community" => $row[1],
                "district" => $row[2],
            ];
            })->toArray();

            DB::table('communities')->insert($records);
        });
    }
}

#6 Run Seeder and Test

Now, run your application using the serve command.

php artisan serve

After that put the following command in your terminal and you can see just 2 or 3 seconds your large number or CSV file will be inserted in database table.

php artisan db:seed --class=CommunitySeeder

I hope you enjoy the Laravel import of large/huge numbers of records CSV file and insert to the database.