Skip to content

HNSW Implementation with Testcases

Mingchung Xia requested to merge hnsw into main

This merge request contains the implementation and test cases of HNSW that I have worked on since January.

Main Changes and Additions

  • Implementation of the HNSW corpuses and vector indexes can be found under ./Sources/SwiftNLP/1. Data Collection and ./Sources/SwiftNLP/1. Data Collection/HNSW
  • Test cases for EphemeralHNSWCorpus and DurableHNSWCorpus can be found under ./Tests/SwiftNLPTests/1. Data Collection/HNSW
  • The sample SwiftNLPVisualizer can be found under ./Sources/SwiftNLPVisualizer; this will be further implemented eventually

Specifications

  • Durable HNSW tests are removed from the GitLab pipeline; changes were made to .gitlab-ci.yml to achieve this
  • Package.swift has additional dependencies for linear algebra and HNSW, in addition to target schemes for SwiftNLP and SwiftNLPVisualizer separation
  • Some legacy files or code are completely commented out
  • Supports both Mac (optimized) and Linux
  • Updated README as needed

Refer to some of the closed issues and some of my notes here for further details. Also refer to the comments and documentation in the code for reference.

Mingchung Xia, W24

Merge request reports