diff --git a/README.md b/README.md index 46af909..b6ac10f 100644 --- a/README.md +++ b/README.md @@ -4,20 +4,6 @@ AutoRAG Dashboard는 실험 결과를 시각화하여 분석할 수 있는 웹 --- -## **🚀 주요 기능** -### ✅ **1. 실험 결과 요약** -- **실험 폴더 (`trial_dir`)를 자동으로 분석**하여 실험 결과를 요약합니다. -- `summary.csv`를 기반으로 가장 성능이 좋은 모듈을 자동으로 선택합니다. -- 실험에 사용된 설정 파일(`config.yaml`)을 Markdown 형식으로 출력합니다. - -### ✅ **2. 실험별 성능 비교** -- Stripplot 및 Boxplot을 제공하여 **모듈별 성능 분포를 시각적으로 비교**할 수 있습니다. -- 성능 비교를 통해 실험 결과를 더욱 직관적으로 이해할 수 있습니다. - -### ✅ **3. 개별 쿼리 조회** -- `detail` 버튼을 클릭하여 쿼리 별로 자세한 정보를를 제공합니다. ---- - ## **📥 설치 및 실행 방법** ### **1️⃣ Docker 컨테이너 실행 (권장)** AutoRAG Dashboard는 Docker 환경에서 쉽게 실행할 수 있습니다. @@ -47,8 +33,7 @@ docker compose up ├── docker-compose.yml # Docker 설정 파일 ├── Dockerfile # Docker 빌드 파일 ├── pyproject.toml # Python 패키지 설정 -├── requirements.txt # 필수 패키지 목록 -└── projects/ # 실험 결과 저장 폴더 (볼륨 마운트) +└── requirements.txt # 필수 패키지 목록 ``` --- @@ -58,13 +43,34 @@ docker compose up 기본적으로 **7690번 포트**에서 실행됩니다. 포트를 변경하려면 `docker-compose.yml`에서 아래 내용을 수정하세요. ```yaml - ports: - - "7690:7690" # 변경하고 싶은 포트로 수정 가능 +# docker-compose.yml +ports: + - "7690:7690" +``` +```shell +# dashboard.sh +python3 -m autorag.cli dashboard \ + --port 7690 ``` ### **📍 실험 데이터 경로 변경** -실험 결과 폴더(`trial_dir`)의 기본 위치는 `./projects/benchmark_sample`입니다. +대시보드 실행 테스트를 위해 `./example` 폴더를 제공합니다. 다른 폴더를 사용하려면 실행 시 `--trial_dir` 옵션 경로를 수정하세요. ```bash -python3 -m autorag.cli dashboard --trial_dir ./projects/custom_experiment -``` \ No newline at end of file +python3 -m autorag.cli dashboard \ + --trial_dir ./projects/custom_experiment +``` + +--- +## **🚀 주요 기능** +### ✅ **1. 실험 결과 요약** +- **실험 폴더 (`trial_dir`)를 자동으로 분석**하여 실험 결과를 요약합니다. +- `summary.csv`를 기반으로 가장 성능이 좋은 모듈을 자동으로 선택합니다. +- 실험에 사용된 설정 파일(`config.yaml`)을 Markdown 형식으로 출력합니다. + +### ✅ **2. 실험별 성능 비교** +- Stripplot 및 Boxplot을 제공하여 **모듈별 성능 분포를 시각적으로 비교**할 수 있습니다. +- 성능 비교를 통해 실험 결과를 더욱 직관적으로 이해할 수 있습니다. + +### ✅ **3. 개별 쿼리 조회** +- `detail` 버튼을 클릭하여 쿼리 별로 자세한 정보를를 제공합니다. \ No newline at end of file diff --git a/dashboard.sh b/dashboard.sh index 23bc431..ad17380 100644 --- a/dashboard.sh +++ b/dashboard.sh @@ -1,7 +1,7 @@ #!/bin/bash python3 -m autorag.cli dashboard \ - --trial_dir /usr/src/app/projects/benchmark_sample/1 \ + --trial_dir /usr/src/app/example/1 \ --port 7690 echo "📊 AutoRAG 대시보드 실행 중..." \ No newline at end of file diff --git a/projects/benchmark_sample/0/config.yaml b/example/0/config.yaml similarity index 100% rename from projects/benchmark_sample/0/config.yaml rename to example/0/config.yaml diff --git a/projects/benchmark_sample/1/config.yaml b/example/1/config.yaml similarity index 100% rename from projects/benchmark_sample/1/config.yaml rename to example/1/config.yaml diff --git a/projects/benchmark_sample/1/post_retrieve_node_line/generator/0.parquet b/example/1/post_retrieve_node_line/generator/0.parquet similarity index 100% rename from projects/benchmark_sample/1/post_retrieve_node_line/generator/0.parquet rename to example/1/post_retrieve_node_line/generator/0.parquet diff --git a/projects/benchmark_sample/1/post_retrieve_node_line/generator/best_0.parquet b/example/1/post_retrieve_node_line/generator/best_0.parquet similarity index 100% rename from projects/benchmark_sample/1/post_retrieve_node_line/generator/best_0.parquet rename to example/1/post_retrieve_node_line/generator/best_0.parquet diff --git a/projects/benchmark_sample/1/post_retrieve_node_line/generator/summary.csv b/example/1/post_retrieve_node_line/generator/summary.csv similarity index 100% rename from projects/benchmark_sample/1/post_retrieve_node_line/generator/summary.csv rename to example/1/post_retrieve_node_line/generator/summary.csv diff --git a/projects/benchmark_sample/1/post_retrieve_node_line/prompt_maker/0.parquet b/example/1/post_retrieve_node_line/prompt_maker/0.parquet similarity index 100% rename from projects/benchmark_sample/1/post_retrieve_node_line/prompt_maker/0.parquet rename to example/1/post_retrieve_node_line/prompt_maker/0.parquet diff --git a/projects/benchmark_sample/1/post_retrieve_node_line/prompt_maker/best_0.parquet b/example/1/post_retrieve_node_line/prompt_maker/best_0.parquet similarity index 100% rename from projects/benchmark_sample/1/post_retrieve_node_line/prompt_maker/best_0.parquet rename to example/1/post_retrieve_node_line/prompt_maker/best_0.parquet diff --git a/projects/benchmark_sample/1/post_retrieve_node_line/prompt_maker/summary.csv b/example/1/post_retrieve_node_line/prompt_maker/summary.csv similarity index 100% rename from projects/benchmark_sample/1/post_retrieve_node_line/prompt_maker/summary.csv rename to example/1/post_retrieve_node_line/prompt_maker/summary.csv diff --git a/projects/benchmark_sample/1/post_retrieve_node_line/summary.csv b/example/1/post_retrieve_node_line/summary.csv similarity index 100% rename from projects/benchmark_sample/1/post_retrieve_node_line/summary.csv rename to example/1/post_retrieve_node_line/summary.csv diff --git a/projects/benchmark_sample/1/retrieve_node_line/passage_reranker/0.parquet b/example/1/retrieve_node_line/passage_reranker/0.parquet similarity index 100% rename from projects/benchmark_sample/1/retrieve_node_line/passage_reranker/0.parquet rename to example/1/retrieve_node_line/passage_reranker/0.parquet diff --git a/projects/benchmark_sample/1/retrieve_node_line/passage_reranker/best_0.parquet b/example/1/retrieve_node_line/passage_reranker/best_0.parquet similarity index 100% rename from projects/benchmark_sample/1/retrieve_node_line/passage_reranker/best_0.parquet rename to example/1/retrieve_node_line/passage_reranker/best_0.parquet diff --git a/projects/benchmark_sample/1/retrieve_node_line/passage_reranker/summary.csv b/example/1/retrieve_node_line/passage_reranker/summary.csv similarity index 100% rename from projects/benchmark_sample/1/retrieve_node_line/passage_reranker/summary.csv rename to example/1/retrieve_node_line/passage_reranker/summary.csv diff --git a/projects/benchmark_sample/1/retrieve_node_line/retrieval/0.parquet b/example/1/retrieve_node_line/retrieval/0.parquet similarity index 100% rename from projects/benchmark_sample/1/retrieve_node_line/retrieval/0.parquet rename to example/1/retrieve_node_line/retrieval/0.parquet diff --git a/projects/benchmark_sample/1/retrieve_node_line/retrieval/1.parquet b/example/1/retrieve_node_line/retrieval/1.parquet similarity index 100% rename from projects/benchmark_sample/1/retrieve_node_line/retrieval/1.parquet rename to example/1/retrieve_node_line/retrieval/1.parquet diff --git a/projects/benchmark_sample/1/retrieve_node_line/retrieval/2.parquet b/example/1/retrieve_node_line/retrieval/2.parquet similarity index 100% rename from projects/benchmark_sample/1/retrieve_node_line/retrieval/2.parquet rename to example/1/retrieve_node_line/retrieval/2.parquet diff --git a/projects/benchmark_sample/1/retrieve_node_line/retrieval/3.parquet b/example/1/retrieve_node_line/retrieval/3.parquet similarity index 100% rename from projects/benchmark_sample/1/retrieve_node_line/retrieval/3.parquet rename to example/1/retrieve_node_line/retrieval/3.parquet diff --git a/projects/benchmark_sample/1/retrieve_node_line/retrieval/4.parquet b/example/1/retrieve_node_line/retrieval/4.parquet similarity index 100% rename from projects/benchmark_sample/1/retrieve_node_line/retrieval/4.parquet rename to example/1/retrieve_node_line/retrieval/4.parquet diff --git a/projects/benchmark_sample/1/retrieve_node_line/retrieval/5.parquet b/example/1/retrieve_node_line/retrieval/5.parquet similarity index 100% rename from projects/benchmark_sample/1/retrieve_node_line/retrieval/5.parquet rename to example/1/retrieve_node_line/retrieval/5.parquet diff --git a/projects/benchmark_sample/1/retrieve_node_line/retrieval/best_2.parquet b/example/1/retrieve_node_line/retrieval/best_2.parquet similarity index 100% rename from projects/benchmark_sample/1/retrieve_node_line/retrieval/best_2.parquet rename to example/1/retrieve_node_line/retrieval/best_2.parquet diff --git a/projects/benchmark_sample/1/retrieve_node_line/retrieval/summary.csv b/example/1/retrieve_node_line/retrieval/summary.csv similarity index 100% rename from projects/benchmark_sample/1/retrieve_node_line/retrieval/summary.csv rename to example/1/retrieve_node_line/retrieval/summary.csv diff --git a/projects/benchmark_sample/1/retrieve_node_line/summary.csv b/example/1/retrieve_node_line/summary.csv similarity index 100% rename from projects/benchmark_sample/1/retrieve_node_line/summary.csv rename to example/1/retrieve_node_line/summary.csv diff --git a/projects/benchmark_sample/1/summary.csv b/example/1/summary.csv similarity index 100% rename from projects/benchmark_sample/1/summary.csv rename to example/1/summary.csv diff --git a/projects/benchmark_sample/data/corpus.parquet b/example/data/corpus.parquet similarity index 100% rename from projects/benchmark_sample/data/corpus.parquet rename to example/data/corpus.parquet diff --git a/projects/benchmark_sample/data/qa.parquet b/example/data/qa.parquet similarity index 100% rename from projects/benchmark_sample/data/qa.parquet rename to example/data/qa.parquet diff --git a/projects/benchmark_sample/resources/bm25_ko_kiwi.pkl b/example/resources/bm25_ko_kiwi.pkl similarity index 100% rename from projects/benchmark_sample/resources/bm25_ko_kiwi.pkl rename to example/resources/bm25_ko_kiwi.pkl diff --git a/projects/benchmark_sample/resources/chroma/985a388c-103b-4534-a1a5-d7088ed74c0c/data_level0.bin b/example/resources/chroma/985a388c-103b-4534-a1a5-d7088ed74c0c/data_level0.bin similarity index 100% rename from projects/benchmark_sample/resources/chroma/985a388c-103b-4534-a1a5-d7088ed74c0c/data_level0.bin rename to example/resources/chroma/985a388c-103b-4534-a1a5-d7088ed74c0c/data_level0.bin diff --git a/projects/benchmark_sample/resources/chroma/985a388c-103b-4534-a1a5-d7088ed74c0c/header.bin b/example/resources/chroma/985a388c-103b-4534-a1a5-d7088ed74c0c/header.bin similarity index 100% rename from projects/benchmark_sample/resources/chroma/985a388c-103b-4534-a1a5-d7088ed74c0c/header.bin rename to example/resources/chroma/985a388c-103b-4534-a1a5-d7088ed74c0c/header.bin diff --git a/projects/benchmark_sample/resources/chroma/985a388c-103b-4534-a1a5-d7088ed74c0c/index_metadata.pickle b/example/resources/chroma/985a388c-103b-4534-a1a5-d7088ed74c0c/index_metadata.pickle similarity index 100% rename from projects/benchmark_sample/resources/chroma/985a388c-103b-4534-a1a5-d7088ed74c0c/index_metadata.pickle rename to example/resources/chroma/985a388c-103b-4534-a1a5-d7088ed74c0c/index_metadata.pickle diff --git a/projects/benchmark_sample/resources/chroma/985a388c-103b-4534-a1a5-d7088ed74c0c/length.bin b/example/resources/chroma/985a388c-103b-4534-a1a5-d7088ed74c0c/length.bin similarity index 100% rename from projects/benchmark_sample/resources/chroma/985a388c-103b-4534-a1a5-d7088ed74c0c/length.bin rename to example/resources/chroma/985a388c-103b-4534-a1a5-d7088ed74c0c/length.bin diff --git a/projects/benchmark_sample/resources/chroma/985a388c-103b-4534-a1a5-d7088ed74c0c/link_lists.bin b/example/resources/chroma/985a388c-103b-4534-a1a5-d7088ed74c0c/link_lists.bin similarity index 100% rename from projects/benchmark_sample/resources/chroma/985a388c-103b-4534-a1a5-d7088ed74c0c/link_lists.bin rename to example/resources/chroma/985a388c-103b-4534-a1a5-d7088ed74c0c/link_lists.bin diff --git a/projects/benchmark_sample/resources/chroma/chroma.sqlite3 b/example/resources/chroma/chroma.sqlite3 similarity index 100% rename from projects/benchmark_sample/resources/chroma/chroma.sqlite3 rename to example/resources/chroma/chroma.sqlite3 diff --git a/projects/benchmark_sample/resources/vectordb.yaml b/example/resources/vectordb.yaml similarity index 100% rename from projects/benchmark_sample/resources/vectordb.yaml rename to example/resources/vectordb.yaml diff --git a/projects/benchmark_sample/trial.json b/example/trial.json similarity index 100% rename from projects/benchmark_sample/trial.json rename to example/trial.json