MB-62577: Merge 'facebookresearch/faiss@v1.8.0' into blevesearch/faiss@bleve

* Merging faiss's [v1.8.0](https://github.com/facebookresearch/faiss/releases/tag/v1.8.0) release into bleve branch on fork.
* Brings in:
```
|\
| * 943d08b Junjie Qi | Prepare for v.1.8.0 release (facebookresearch#3265)
| * abff75e Jeff Johnson | faiss gpu: fix DeviceVector reallocations (facebookresearch#3256)
| * b8d91d8 Gergely Szilvasy | fixes for D51074065
| * 27b1055 Tarang Jain | Integrate IVF-PQ from RAFT (facebookresearch#3044)
| * 87d43b9 Jim Borden | Fix AVX2 build on Windows (facebookresearch#3238)
| * c577f43 Gergely Szilvasy | fix perf test (facebookresearch#3255)
| * 8400ece Richard Barnes | Remove unused variables in faiss/utils/partitioning.cpp
| * 873b1bc Gergely Szilvasy | fix perf test (facebookresearch#3253)
| * 6d00c41 Richard Barnes | Remove unused variables in faiss/utils/hamming_distance/neon-inl.h
| * c0624d2 Richard Barnes | Remove unused variables in faiss/invlists/OnDiskInvertedLists.cpp
| * a87b432 Richard Barnes | Remove unused variables in faiss/IndexIVFPQ.cpp
| * 1338e0d Richard Barnes | Remove unused variables in faiss/IndexBinaryIVF.cpp
| * 1b0d274 Richard Barnes | Remove unused variables in faiss/IndexIVF.cpp
| * 53fc617 Richard Barnes | Remove unused variables in faiss/tests/test_threaded_index.cpp
| * aaca1c2 Richard Barnes | Remove semicolon(s) from 3 files inc facer/engine/utils/UDPServer.h
| * a187394 Gergely Szilvasy | libraft 24.02 is released, switching channel
| * f262011 Gergely Szilvasy | fix omp parallelism in fast scan range search
| * 8898eab yuhaijun999 | Add feature in IndexIDMap.cpp range_search with Parameters. (facebookresearch#3213)
| * ebb5f84 Xiaozhong Pan | add a context parameter to InvertedLists and InvertedListsIterator (facebookresearch#3247)
| * bfa46a3 Gergely Szilvasy | pin circleci windows image (facebookresearch#3248)
| * ed3f6e5 Gergely Szilvasy | faiss-gpu to use 11.4.4 and be the only one supporting P100 (facebookresearch#3237)
| * 31a29d0 xinhuitian | HNSW search use efSearch from params if provided (facebookresearch#3233)
| * c1822a8 Gergely Szilvasy | cuda dependencies (facebookresearch#3230)
| * 1d0e8d4 Gergely Szilvasy | index optimizer (facebookresearch#3154)
| * 75ae0bf Richard Barnes | Remove unused variables in faiss/IndexScalarQuantizer.cpp
| * 420d25f Maria Lomeli | Index pretransform support in search_preassigned (facebookresearch#3225)
| * 31bddeb Gergely Szilvasy | add faiss_gpu lib (facebookresearch#3222)
| * 2817344 Gergely Szilvasy | fix ACCESS VIOLATION error when searching using IDSelectorArray
| * 67c6a19 Richard Barnes | Remove extra semi colon from faiss/impl/ProductAdditiveQuantizer.h
| * 898ce35 Richard Barnes | Remove unused function from faiss/utils/distances_simd.cpp
| * 1dcb5d3 Kota Yamaguchi | Add missing header files (facebookresearch#3218)
| * 51b6083 Gergely Szilvasy | faiss on rocksdb demo (facebookresearch#3216)
| * c4b91a5 Gergely Szilvasy | Replace pickle serialization to address security vulnerability
| * a30fd74 Richard Barnes | Remove unused function from faiss/clone_index.cpp
| * fc4dbd5 Richard Barnes | Remove extra semi colon from faiss/IndexIVFSpectralHash.cpp
| * 6b2c79c Richard Barnes | Remove extra semi colon from faiss/index_factory.cpp
| * a7b76a7 Richard Barnes | Fix shadowed variable in faiss/IndexAdditiveQuantizerFastScan.cpp
| * ae25b1b Richard Barnes | Fix shadowed variable in faiss/impl/residual_quantizer_encode_steps.cpp
| * 7c4fb6d Richard Barnes | Fix shadowed variable in faiss/impl/ResultHandler.h
| * 12637a2 Richard Barnes | Fix shadowed variable in faiss/utils/utils.cpp
| * 63edc32 Kota Yamaguchi | Fix AVX512 build targets in CMakeLists.txt (facebookresearch#3214)
| * 99c0e2b Richard Barnes | Remove unused variables in faiss/utils/sorting.cpp
| * e8494f6 Richard Barnes | Remove unused variables in faiss/IndexBinaryHash.cpp
| * bbb6ec7 Richard Barnes | Remove unused variables in faiss/IndexShardsIVF.cpp
| * 0716bde Richard Barnes | Fix shadowed variable in faiss/IndexNSG.cpp
| * bffedda Richard Barnes | Fix shadowed variable in faiss/IndexIDMap.cpp
| * a70a8a5 Richard Barnes | Remove unused variables in faiss/impl/HNSW.cpp
| * b274cb4 Richard Barnes | Remove unused variables in faiss/IndexIVFFastScan.cpp
| * e55a0ac Richard Barnes | Remove unused variables in faiss/impl/index_write.cpp
| * a651069 Richard Barnes | Remove unused variables in faiss/impl/index_read.cpp
| * e19de27 Richard Barnes | Remove extra semi colon from faiss/IndexIVFFastScan.cpp
| * b10eb35 Richard Barnes | Remove extra semi colon from faiss/Clustering.cpp
| * 683eadf Richard Barnes | Remove extra semi colon from faiss/utils/distances_simd.cpp
| * 522452f Richard Barnes | Remove extra semi colon from faiss/python/python_callbacks.cpp
| * 3e666ae Richard Barnes | Remove extra semi colon from faiss/invlists/InvertedLists.cpp
| * 7e01b47 Gergely Szilvasy | fix raft build
| * 7f3e0a3 Richard Barnes | Mismatch tags take 2 (facebookresearch#3211)
| * aff812e Richard Barnes | Fix shadowed variable in faiss/clone_index.cpp
| * c540e76 Kota Yamaguchi | Use packaging instead of deprecated distutils for version handling (facebookresearch#3191)
| * 091f344 Gergely Szilvasy | add gpu to avx512 (facebookresearch#3210)
| * 5e3eae4 Richard Barnes | Remove unused variables in faiss/utils/hamming.cpp
| * 739ce13 Matthijs Douze | doc update to mention the paper
| * 9a63a3c Richard Barnes | Fix shadowed variable in faiss/IndexFastScan.cpp
| * 7dd06dd Richard Barnes | Fix shadowed variable in faiss/impl/NNDescent.cpp
| * 0fc8456 Maria Lomeli | Offline IVF powered by faiss big batch search (facebookresearch#3202)
| * 4150fb1 Richard Barnes | Fix shadowed variable in faiss/impl/PolysemousTraining.cpp
| * 3973017 Richard Barnes | Fix shadowed variable in faiss/IndexBinaryIVF.cpp
| * 46320e0 Richard Barnes | Fix shadowed variable in faiss/utils/hamming_distance/avx2-inl.h
| * 1be1d32 Richard Barnes | Fix shadowed variable in faiss/impl/io.cpp
| * 7442a54 Richard Barnes | Fix shadowed variable in faiss/impl/index_write.cpp
| * cfcefc0 Richard Barnes | Remove unused variables in faiss/utils/sorting.cpp
| * f884ba2 Richard Barnes | Remove unused variables in faiss/IndexHNSW.cpp
| * b7efd3a Gergely Szilvasy | bump libraft version to fix nightly
| * c3aa526 Naveen Tatikonda | Add SIMD NEON Optimization for QT_FP16 in Scalar Quantizer (facebookresearch#3166)
| * 32f0e8c Matthijs Douze | Generalize ResultHanlder, support range search for HNSW and Fast Scan (facebookresearch#3190)
| * 0013c70 Gergely Szilvasy | avx512 CI + conda packages (facebookresearch#3197)
| * 0710cbd Richard Barnes | Fix shadowed variable in faiss/impl/ResultHandler.h
| * 42b6216 Richard Barnes | Fix shadowed variable in faiss/impl/ScalarQuantizer.cpp
| * c399b11 Richard Barnes | Fix shadowed variable in faiss/utils/hamming.cpp
| * 952941b Richard Barnes | Fix shadowed variable in faiss/MatrixStats.cpp
| * db09984 Richard Barnes | Fix shadowed variable in faiss/impl/simd_result_handlers.h
| * beef610 Gergely Szilvasy | faiss paper benchmarks (facebookresearch#3189)
| * b7681be Richard Barnes | Remove unused exception parameter from files inc facer/engine/utils/StatsD.cpp
| * 449f02c Richard Barnes | Add missing fallthrough to fbcode/faiss/utils/hamming_distance/neon-inl.h
| * 0727fff Eddy Li | Adding [[fallthrough]] annotation to case statements in faiss/utils/hamming_distance/neon-inl.h
| * 9d74f09 Jason Sylka | Adding [[fallthrough]] annotation to case statements in faiss/utils/hamming_distance/generic-inl.h
| * 77c28f8 Gergely Szilvasy | Back out "Offline IVF powered by faiss big batch search"
| * 65b08ef Richard Barnes | Remove unused function from faiss/utils/quantize_lut.cpp
| * 5621487 Richard Barnes | Remove unused function from faiss/impl/ProductQuantizer.cpp
| * 9a8b34e Maria | Offline IVF powered by faiss big batch search (facebookresearch#3175)
| * be12427 Maria Lomeli | Upstream changes to big batch search (facebookresearch#3170)
| * 79f558f Xiaozhong Pan | set keep_max for InvertedListScanner child classes (facebookresearch#3169)
| * 9f8f238 Bader-eddine Ouaich | Fix potential buffer overflow (facebookresearch#3146)
| * 9107035 Jaepil Jeong | Fix compile error (facebookresearch#3162)
| * d40b39a Alexandr Guzhva | Nit: fix variable types (facebookresearch#3147)
| * aebc1d5 Richard Barnes | Remove extra semi colon from faiss/impl/IDSelector.h
| * 3dab47c Richard Barnes | Remove extra semi colon from facer/engine/libsvm/svm.cpp
| * d2cb016 Richard Barnes | Remove unused exception parameter from files inc faiss/tests/test_approx_topk.cpp
| * 5b6c4b4 Matthijs Douze | Back out "printf -> fmt::print in files inc faiss/IndexBinaryHNSW.cpp" (facebookresearch#3164)
| * 131adc5 Richard Barnes | printf -> fmt::print in files inc faiss/IndexBinaryHNSW.cpp
| * eefa391 Alexandr Guzhva | Introduce avx512 optimization mode and FAISS_OPT_LEVEL env variable (facebookresearch#3150)
| * 4c83965 Gergely Szilvasy | benchmark view results (facebookresearch#3144)
| * 9519a19 Gergely Szilvasy | benchmark refactor
| * a5b03cb Alexandr Guzhva | Fix build on Alpine Linux (facebookresearch#3148)
| * 4bf8f93 Yuri Vanin | Add NegativeDistanceComputer::distances_batch_4 override (facebookresearch#3149)
| * 90654d6 Gergely Szilvasy | benchmark core faiss prereqs
| * 04bb0a8 Alexandr Guzhva | improve ScalarQuantizer performance, ESPECIALLY on old GCC (facebookresearch#3141)
| * 43f8220 Matthijs Douze | fix scopedeleter diff
| * d3692d2 Alexandr Guzhva | Deprecate ScopeDeleter and ScopeDeleter1 in favor of std::unique_ptr<[]> (facebookresearch#3108)
| * eb071f8 luyuncheng | Fix is_trained in IndexNSGSQ (facebookresearch#3145)
| * d643c41 Ben Frederickson | use precomputed norms for raft brute_force knn calls (facebookresearch#3089)
| * b109d08 Matthijs Douze | Search and return codes (facebookresearch#3143)
| * 467f70e Jeff Johnson | Consolidate GPU IVF query tile calculation + special handling for large query memory requirements
| * 411c172 Robert Maynard | Add linker script to support large cuda cubin files (facebookresearch#3115)
| * 09c7aac Jeff Johnson | Faiss GPU CUDA 12 fix: warp synchronous behavior
| * 0c2243c Christopher Ponce de Leon | Revert D51029740: Namespace doesn't need to be followed by semicolon
| * 438b519 Richard Barnes | Namespace doesn't need to be followed by semicolon
| * 9a66532 Alexandr Guzhva | Add search parameters for IndexRefine::search() and IndexRefineFlat::search() (facebookresearch#3122)
| * df7280b pe4eniks | Documentation fixes (facebookresearch#3092)
| * 6b76150 chasingegg | Remove confusing comments in partitioning.cpp (facebookresearch#3104)
| * 6c89c8b Gergely Szilvasy | fix faiss-gpu nightly
| * 0c07a11 Gergely Szilvasy | fix raft contbuild and switch to libraft 23.12 (facebookresearch#3116)
| * 9bb6b4b Gergely Szilvasy | fix test TestCrossCodebookComputations::test_precomp
| * c3b9374 Gergely Szilvasy | bench_fw - fixes & nits for oss (facebookresearch#3102)
| * 0a00d81 Gergely Szilvasy | offline index evaluation (facebookresearch#3097)
| * f969d7a Matthijs Douze | better docs
| * edcf743 Corey J. Nolet | Integrate IVF-Flat from RAFT (facebookresearch#2521)
| * 458633c Robert Maynard | Remove uneeded PTX code generation from libfaiss builds (facebookresearch#3083)
| * 2b48901 Matthijs Douze | Remove 1L and 1UL
| * 3f3321c Alexandr Guzhva | Small refactoring of inverted lists (facebookresearch#3055)
| * 834c543 Haijun Yu | Fix SingleBestResultHandler bug. if IndexFlatL2 empty data then search topk = 1 return label = 0 not -1. (facebookresearch#3075)
| * 9db1824 Matthijs Douze | Relax IVFFlatDedup test (facebookresearch#3077)
| * 0f18251 generatedunixname89002005325676 | Daily `arc lint --take CLANGFORMAT`
| * cf90435 Matthijs Douze | fix flaky GPU test
| * e18de23 Alexandr Guzhva | Fix chunk-based processing in ResidualCoarseQuantizer::search() (facebookresearch#3047)
| * a1814be Alexandr Guzhva | Simplify dependency components chain (facebookresearch#3058)
| * 56b1087 Alexandr Guzhva | move fvec_madd_* functions declarations to a right header (facebookresearch#3054)
| * 0780a28 Alexandr Guzhva | Fix a couple of type mismatches (facebookresearch#3059)
| * 592f301 Alexandr Guzhva | Unneeded field, exists in a baseclass (facebookresearch#3064)
| * 6218111 chasingegg | Fix some typos (facebookresearch#3056)
| * d85601d generatedunixname89002005287564 | fairring, faiss, fairness (4401366386162573988)
| * 50be4ea generatedunixname89002005287564 | faiss, falcon (1203443027085661913)
| * c8d6f7b Matthijs Douze | fix CI issues after cross-matrix diff (facebookresearch#3042)
| * 4699365 Naveen Tatikonda | Fix is_trained in IndexHNSWSQ (facebookresearch#3034)
| * 1d6db93 generatedunixname89002005325676 | Daily `arc lint --take CLANGFORMAT`
| * 9dc75d0 Matthijs Douze | reduce cross table size (facebookresearch#3012)
| * 039409d Matthijs Douze | split off RQ encoding steps to another file (facebookresearch#3011)
| * 67d8727 Matthijs Douze | Clean up batch comments + obey IO_FLAG_SKIP_PRECOMPUTE_TABLE (facebookresearch#3013)
| * 82352dd Matthijs Douze | make nbits configurable for graph indices based on PQ (facebookresearch#3031)
| * 5c4bd3f Matthijs Douze | Cleanup clustering code (facebookresearch#3030)
| * 3888f9b Corey J. Nolet | Using expanded distance forms in `RaftFlatIndex.cu` (facebookresearch#3021)
| * fef49a6 Richard Barnes | Del `(object)` from 50 inc faic/experiments/blip_finetune/transform/randaugment.py
| * c00fe25 Gergely Szilvasy | faiss-gpu-raft, fix dispatch test (facebookresearch#3017)
| * a02b37d Gergely Szilvasy | relax test_lut rtol (facebookresearch#3016)
| * 69cb877 Matthijs Douze | Fix memory leak for ParameterSpace objects (facebookresearch#3007)
| * e3731f7 Gergely Szilvasy | faiss-gpu-raft, the missing bits (facebookresearch#3009)
| * 88b7255 qmc20234 | fix argument error (facebookresearch#2965)
| * 2768fb3 Gergely Szilvasy | faiss-gpu-raft package (facebookresearch#2992)
| * c09992b Maria Lomeli | Back out "Better NaN handling" (facebookresearch#3006)
| * e3deb71 Fernando Gasperi | Enable for faiss tests (facebookresearch#3002)
| * ef7e945 Gergely Szilvasy | remove avx2 from raft cmake contbuild
| * 687457b Matthijs Douze | Access graph structure for NSG (facebookresearch#2984)
| * da16d9d Gergely Szilvasy | simplify raft build (facebookresearch#2983)
| * a3fbf2d Matthijs Douze | Better NaN handling (facebookresearch#2986)
| * a4ddb18 generatedunixname89002005325676 | Daily `arc lint --take CLANGFORMAT`
| * 35dac92 Maria | Added version to nighly install (facebookresearch#2982)
| * 5a95d47 Alexandr Guzhva | Upgrade AVX2 code for SQ8 (facebookresearch#2942)
| * 0aae4d3 youcheng huang | fix hnsw shrink_neighbor_list comment (facebookresearch#2980)
| * 7bf7149 Corey J. Nolet | Adding `libraft` dependency to speed up compile times with `USE_RAFT` (facebookresearch#2958)
| * 726143d Gergely Szilvasy | install libraft for cmake build (facebookresearch#2968)
| * 821a401 Gergely Szilvasy | CodeSet for deduping large datasets (facebookresearch#2949)
| * 43d86e3 Matthijs Douze | Relax IVF AQ FastScan (facebookresearch#2940)
| * a757806 Maria | added blas=1.0=mkl to INSTALL (facebookresearch#2939)
| * d48e777 Sid Jha | Fix import (facebookresearch#2936)
| * 1c1d5c8 Matthijs Douze | Make tests a little less verbose
| * 4bfdd43 Richard Barnes | Parallelize kernel compilation in FAISS (facebookresearch#2922)
| * a91a288 Matthijs Douze | use dispatcher function to call HammingComputer (facebookresearch#2918)
| * a27036a Matthijs Douze | add small benchmark for hamming computers
| * 391601d Gergely Szilvasy | relax test_ivf_train_2level threshold (facebookresearch#2927)
| * 1d7c05d Gergely Szilvasy | raft nightly (facebookresearch#2926)
| * 9126f86 Octavian Guzu | Prevent snprintf vulnerability
| * 8ac4e41 Richard Barnes | Switch //faiss/gpu to use templates instead of macros (facebookresearch#2914)
| * e0741ca Gergely Szilvasy | fix for lib/jvm/languages/python/bin/conda no such file (facebookresearch#2917)
| * f69b1db Gergely Szilvasy | update installation instructions with notes about mkl and the nvidia channel
| * 07fe2b6 Matthijs Douze | Binary cloning and GPU range search (facebookresearch#2916)
| * e153cac Gergely Szilvasy | fix the osx nightly build (facebookresearch#2896)
| * 092606b Gergely Szilvasy | bbs producer/consumer threading (facebookresearch#2901)
| * d8a6350 I | Update docs (C++11 -> C++17) (facebookresearch#2907)
| * 6951466 Gergely Szilvasy | raft enabled cmake build (facebookresearch#2898)
| * 27ffd14 Richard Barnes | Use C++17 [[fallthrough]] in faiss/utils/distances_simd.cpp (facebookresearch#2913)
| * 100beb8 Richard Barnes | Use C++17 [[fallthrough]] in faiss/utils/hamming_distance/avx2-inl.h
| * 463ffd8 Richard Barnes | Indicate that fallthrough is intentional in faiss (facebookresearch#2897)
| * 8ec166c Taras Tsugrii | Simplify non-optimal points removal.
| * f82298f Taras Tsugrii | Remove unused unordered_map include. (facebookresearch#2900)
| * 451f6cd Gergely Szilvasy | c++ 17 (facebookresearch#2899)
| * 9c88422 I | Some changes to simdlib (facebookresearch#2885)
| * bbc95b1 I | Fix windows CI (facebookresearch#2889)
| * 90349f2 Matthijs Douze | Large two-level clustering (facebookresearch#2882)
| * 6fd0cb6 Alexandr Guzhva | fix a typo (facebookresearch#2881)
| * e8b7575 Alexandr Guzhva | AVX2 version of faiss::HNSW::MinimaxHeap::pop_min() (facebookresearch#2874)
| * 6800ebe Matthijs Douze | Support independent IVF coarse quantizer
| * a3296f4 Alexandr Guzhva | Use uint8_t instead of uint32_t for faiss::VisitedTable.visno (facebookresearch#2873)
| * fd09e51 Matthijs Douze | move by_residual to IndexIVF (facebookresearch#2870)
| * 1c1879b Gergely Szilvasy | tiling bfKnn (facebookresearch#2865)
| * 5c221ed generatedunixname89002005325676 | Daily `arc lint --take CLANGFORMAT`
| * a878c79 Matthijs Douze | Support RAFT from python (facebookresearch#2864)
| * 48d48a3 Matthijs Douze | fix windows test (facebookresearch#2862)
| * 615e3fc generatedunixname89002005325676 | Daily `arc lint --take CLANGFORMAT`
| * b9ea339 Matthijs Douze | support range search from GPU (facebookresearch#2860)
| * 54d331e Matthijs Douze | Update INSTALL.md (facebookresearch#2853)
| * f809cf0 Gergely Szilvasy | fix osx openmp (facebookresearch#2857)
| * eea4a48 Gergely Szilvasy | pin mkl to 2023 (facebookresearch#2856)
| * 81f2709 Alexandr Guzhva | enable RAFT under the hood of GPU FAISS (facebookresearch#2840)
| * 3219e3d Matthijs Douze | Support Selector for IDMap (facebookresearch#2848)
| * 5b17225 Alexandr Guzhva | HNSW speedup + Distance 4 points (facebookresearch#2841)
| * f276c47 Alexandr Guzhva | Minor speedup of HNSW (facebookresearch#2832)
| * 29318b3 Alexandr Guzhva | improve PQ/IVFPQ code_distance() for avx2 for 4 and 8 subquantizers (facebookresearch#2831)
| * bc12015 Alexandr Guzhva | Fix PR problems (facebookresearch#2839)
| * 1fb6968 Alexandr Guzhva | upgrade raft version to 23.06 (facebookresearch#2847)
| * 2d8886c Matthijs Douze | IVF sorting routine (facebookresearch#2846)
| * 4f574d5 Corey J. Nolet | Integrate RAFT FlatIndex / Distances (facebookresearch#2707)
| * cbff63e Gergely Szilvasy | addn_query_subset_with_ids float index bug (facebookresearch#2834)
| * 155065c Gergely Szilvasy | stop expensive osx arm64 builds on every diff (facebookresearch#2835)
| * 3704bbe Matthijs Douze | Add GIST1M to datasets
| * 1cb1e54 Alexandr Guzhva | upgrade horizontal sum in distance_single_code for PQ/IVFPQ (facebookresearch#2830)
| * d0ba4c0 Alexandr Guzhva | Faster versions of fvec_op_ny_Dx for AVX2 (facebookresearch#2811)

```

Author: abhinavdangeti

.circleci/config.yml

@@ -9,19 +9,14 @@ executors:
     environment:
       CONDA_ARCH: Linux-x86_64
     machine:
-      image: linux-cuda-11:2023.02.1
+      image: linux-cuda-12:default
     resource_class: gpu.nvidia.medium
   linux-arm64-cpu:
     environment:
       CONDA_ARCH: Linux-aarch64
     machine:
-      image: ubuntu-2004:current
+      image: ubuntu-2204:current
     resource_class: arm.medium
-  macosx-x86_64-cpu:
-    environment:
-      CONDA_ARCH: MacOSX-x86_64
-    macos:
-      xcode: 11.7.0  # max supported for conda build, https://circleci.com/docs/using-macos#supported-xcode-versions
   macosx-arm64-cpu:
     environment:
       CONDA_ARCH: MacOSX-arm64
@@ -30,7 +25,7 @@ executors:
     resource_class: macos.m1.large.gen1
   windows-x86_64-cpu:
     machine:
-      image: windows-server-2019-vs2019:stable
+      image: windows-server-2019-vs2019:2023.04.1
       shell: bash.exe
     resource_class: windows.medium
 
@@ -66,6 +61,9 @@ jobs:
       cuda:
         type: string
         default: ""
+      raft:
+        type: string
+        default: ""
       cuda_archs:
         type: string
         default: ""
@@ -93,6 +91,8 @@ jobs:
       - run:
           name: Install conda build tools
           command: |
+            # conda config --set solver libmamba
+            # conda config --set verbosity 3
             conda update -y -q conda
             conda install -y -q conda-build
       - when:
@@ -105,14 +105,16 @@ jobs:
                   conda config --set anaconda_upload yes
       - when:
           condition:
-              not: << parameters.label >>
+            and:
+              - not: << parameters.label >>
+              - not: << parameters.cuda >>
           steps:
             - run:
                 name: Conda build (CPU)
                 no_output_timeout: 30m
                 command: |
                   cd conda
-                  conda build faiss --python 3.10 -c pytorch -c pkgs/main -c conda-forge
+                  conda build faiss --python 3.11 -c pytorch
       - when:
           condition:
             and:
@@ -124,21 +126,63 @@ jobs:
                 no_output_timeout: 30m
                 command: |
                   cd conda
-                  conda build faiss --user pytorch --label <<parameters.label>> -c pytorch -c pkgs/main -c conda-forge
+                  conda build faiss --user pytorch --label <<parameters.label>> -c pytorch
+      - when:
+          condition:
+            and:
+              - not: << parameters.label >>
+              - << parameters.cuda >>
+              - not: << parameters.raft >>
+          steps:
+            - run:
+                name: Conda build (GPU)
+                no_output_timeout: 60m
+                command: |
+                  cd conda
+                  conda build faiss-gpu --variants '{ "cudatoolkit": "<<parameters.cuda>>", "c_compiler_version": "<<parameters.compiler_version>>", "cxx_compiler_version": "<<parameters.compiler_version>>" }' \
+                      -c pytorch -c nvidia/label/cuda-<<parameters.cuda>> -c nvidia
       - when:
           condition:
             and:
               - << parameters.label >>
               - << parameters.cuda >>
+              - not: << parameters.raft >>
           steps:
             - run:
                 name: Conda build (GPU) w/ anaconda upload
                 no_output_timeout: 60m
                 command: |
-                  sudo update-alternatives --set cuda /usr/local/cuda-<<parameters.cuda>>
                   cd conda
                   conda build faiss-gpu --variants '{ "cudatoolkit": "<<parameters.cuda>>", "c_compiler_version": "<<parameters.compiler_version>>", "cxx_compiler_version": "<<parameters.compiler_version>>" }' \
-                      --user pytorch --label <<parameters.label>> -c pytorch -c nvidia -c pkgs/main -c conda-forge
+                      --user pytorch --label <<parameters.label>> -c pytorch -c nvidia/label/cuda-<<parameters.cuda>> -c nvidia
+      - when:
+          condition:
+            and:
+              - not: << parameters.label >>
+              - << parameters.cuda >>
+              - << parameters.raft >>
+          steps:
+            - run:
+                name: Conda build (GPU w/ RAFT)
+                no_output_timeout: 60m
+                command: |
+                  cd conda
+                  conda build faiss-gpu-raft --variants '{ "cudatoolkit": "<<parameters.cuda>>", "c_compiler_version": "<<parameters.compiler_version>>", "cxx_compiler_version": "<<parameters.compiler_version>>" }' \
+                      -c pytorch -c nvidia/label/cuda-<<parameters.cuda>> -c nvidia -c rapidsai -c conda-forge
+      - when:
+          condition:
+            and:
+              - << parameters.label >>
+              - << parameters.cuda >>
+              - << parameters.raft >>
+          steps:
+            - run:
+                name: Conda build (GPU w/ RAFT) w/ anaconda upload
+                no_output_timeout: 60m
+                command: |
+                  cd conda
+                  conda build faiss-gpu-raft --variants '{ "cudatoolkit": "<<parameters.cuda>>", "c_compiler_version": "<<parameters.compiler_version>>", "cxx_compiler_version": "<<parameters.compiler_version>>" }' \
+                      --user pytorch --label <<parameters.label>> -c pytorch -c nvidia/label/cuda-<<parameters.cuda>> -c nvidia -c rapidsai -c conda-forge
 
   build_cmake:
     parameters:
@@ -150,6 +194,9 @@ jobs:
       gpu:
         type: string
         default: "OFF"
+      raft:
+        type: string
+        default: "OFF"
     executor: << parameters.exec >>
     environment:
       OMP_NUM_THREADS: 10
@@ -165,18 +212,37 @@ jobs:
               bash miniconda.sh -b -p $HOME/miniconda
               ~/miniconda/bin/conda init
             fi
-      - when:
-          condition:
-            equal: [ "ON", << parameters.gpu >> ]
-          steps:
-            - run:
-                name: Configure CUDA
-                command: sudo update-alternatives --set cuda /usr/local/cuda-11.4
       - run:
           name: Set up environment
           command: |
+            conda config --set solver libmamba
             conda update -y -q conda
-            conda install -y -q cmake=3.23.1 make swig mkl=2021 mkl-devel=2021 numpy scipy pytest gxx_linux-64 sysroot_linux-64=2.17 -c pkgs/main -c conda-forge
+      - when:
+          condition:
+            equal: [ "OFF", << parameters.raft >> ]
+          steps:
+            - run:
+                name: Install env using main channel
+                command: |
+                  conda install -y -q python=3.11 cmake make swig=4.0.2 mkl=2023 mkl-devel=2023 numpy scipy pytest gxx_linux-64 sysroot_linux-64
+      - when:
+          condition:
+            equal: [ "ON", << parameters.raft >> ]
+          steps:
+            - run:
+                name: Install env using conda-forge channel
+                command: |
+                  conda install -y -q python=3.11 cmake make swig=4.0.2 mkl=2023 mkl-devel=2023 numpy scipy pytest gxx_linux-64 sysroot_linux-64=2.28 libraft cuda-version=11.8 cuda-toolkit -c rapidsai-nightly -c "nvidia/label/cuda-11.8.0" -c conda-forge
+      - when:
+          condition:
+            and:
+              - equal: [ "ON", << parameters.gpu >> ]
+              - equal: [ "OFF", << parameters.raft >> ]
+          steps:
+            - run:
+                name: Install CUDA
+                command: |
+                  conda install -y -q cuda-toolkit -c "nvidia/label/cuda-11.8.0"
       - run:
           name: Build all targets
           no_output_timeout: 30m
@@ -185,8 +251,9 @@ jobs:
             conda activate
             cmake -B build \
                   -DBUILD_TESTING=ON \
-                  -DBUILD_SHARED_LIBS=OFF \
+                  -DBUILD_SHARED_LIBS=ON \
                   -DFAISS_ENABLE_GPU=<< parameters.gpu >> \
+                  -DFAISS_ENABLE_RAFT=<< parameters.raft >> \
                   -DFAISS_OPT_LEVEL=<< parameters.opt_level >> \
                   -DFAISS_ENABLE_C_API=ON \
                   -DPYTHON_EXECUTABLE=$(which python) \
@@ -222,7 +289,7 @@ jobs:
             - run:
                 name: Python tests (CPU + GPU)
                 command: |
-                  conda install -y -q pytorch pytorch-cuda -c pytorch -c nvidia
+                  conda install -y -q pytorch pytorch-cuda=11.8 -c pytorch -c nvidia/label/cuda-11.8.0
                   pytest --junitxml=test-results/pytest/results.xml tests/test_*.py
                   pytest --junitxml=test-results/pytest/results-torch.xml tests/torch_*.py
                   cp tests/common_faiss_tests.py faiss/gpu/test
@@ -253,26 +320,29 @@ workflows:
           name: Linux x86_64 AVX2 (cmake)
           exec: linux-x86_64-cpu
           opt_level: "avx2"
+      - build_cmake:
+          name: Linux x86_64 AVX512 (cmake)
+          exec: linux-x86_64-cpu
+          opt_level: "avx512"
       - build_cmake:
           name: Linux x86_64 GPU (cmake)
           exec: linux-x86_64-gpu
           gpu: "ON"
           requires:
-            - Linux x86_64 (cmake)
+            - Linux x86_64 AVX2 (cmake)
+      - build_cmake:
+          name: Linux x86_64 GPU w/ RAFT (cmake)
+          exec: linux-x86_64-gpu
+          gpu: "ON"
+          raft: "ON"
+          requires:
+            - Linux x86_64 GPU (cmake)
       - build_conda:
           name: Linux x86_64 (conda)
           exec: linux-x86_64-cpu
-      - build_conda:
-          name: OSX x86_64 (conda)
-          exec: macosx-x86_64-cpu
       - build_conda:
           name: Windows x86_64 (conda)
           exec: windows-x86_64-cpu
-      - build_conda:
-          name: OSX arm64 (conda)
-          exec: macosx-arm64-cpu
-          requires:
-            - Linux arm64 (conda)
       - build_conda:
           name: Linux arm64 (conda)
           exec: linux-arm64-cpu
@@ -286,29 +356,58 @@ workflows:
             branches:
               ignore: /.*/
       - build_conda:
-          name: Linux x86_64 GPU packages (CUDA 11.4)
+          name: Linux x86_64 GPU packages (CUDA 11.4.4)
           exec: linux-x86_64-gpu
           label: main
-          cuda: "11.4"
-          cuda_archs: "60;61;70;72;75;80;86"
+          cuda: "11.4.4"
+          cuda_archs: "60-real;61-real;62-real;70-real;72-real;75-real;80;86-real"
           compiler_version: "11.2"
           filters:
             tags:
               only: /^v.*/
             branches:
               ignore: /.*/
       - build_conda:
-          name: Windows x86_64 packages
-          exec: windows-x86_64-cpu
+          name: Linux x86_64 GPU w/ RAFT packages (CUDA 11.8.0)
+          exec: linux-x86_64-gpu
           label: main
+          raft: "ON"
+          cuda: "11.8.0"
+          cuda_archs: "70-real;72-real;75-real;80;86-real"
+          compiler_version: "11.2"
           filters:
             tags:
               only: /^v.*/
             branches:
               ignore: /.*/
       - build_conda:
-          name: OSX x86_64 packages
-          exec: macosx-x86_64-cpu
+          name: Linux x86_64 GPU packages (CUDA 12.1.1)
+          exec: linux-x86_64-gpu
+          label: main
+          cuda: "12.1.1"
+          cuda_archs: "70-real;72-real;75-real;80;86-real"
+          compiler_version: "11.2"
+          filters:
+            tags:
+              only: /^v.*/
+            branches:
+              ignore: /.*/
+      - build_conda:
+          name: Linux x86_64 GPU w/ RAFT packages (CUDA 12.1.1)
+          exec: linux-x86_64-gpu
+          label: main
+          raft: "ON"
+          cuda: "12.1.1"
+          cuda_archs: "70-real;72-real;75-real;80;86-real"
+          compiler_version: "11.2"
+          filters:
+            tags:
+              only: /^v.*/
+            branches:
+              ignore: /.*/
+      - build_conda:
+          name: Windows x86_64 packages
+          exec: windows-x86_64-cpu
           label: main
           filters:
             tags:
@@ -348,19 +447,38 @@ workflows:
           exec: linux-x86_64-cpu
           label: nightly
       - build_conda:
-          name: Linux x86_64 GPU nightlies (CUDA 11.4)
+          name: Linux x86_64 GPU nightlies (CUDA 11.4.4)
+          exec: linux-x86_64-gpu
+          label: nightly
+          cuda: "11.4.4"
+          cuda_archs: "60-real;61-real;62-real;70-real;72-real;75-real;80;86-real"
+          compiler_version: "11.2"
+      - build_conda:
+          name: Linux x86_64 GPU w/ RAFT nightlies (CUDA 11.8.0)
           exec: linux-x86_64-gpu
-          cuda: "11.4"
-          cuda_archs: "60;61;70;72;75;80;86"
+          label: nightly
+          raft: "ON"
+          cuda: "11.8.0"
+          cuda_archs: "70-real;72-real;75-real;80;86-real"
           compiler_version: "11.2"
+      - build_conda:
+          name: Linux x86_64 GPU nightlies (CUDA 12.1.1)
+          exec: linux-x86_64-gpu
           label: nightly
+          cuda: "12.1.1"
+          cuda_archs: "70-real;72-real;75-real;80;86-real"
+          compiler_version: "11.2"
       - build_conda:
-          name: Windows x86_64 nightlies
-          exec: windows-x86_64-cpu
+          name: Linux x86_64 GPU w/ RAFT nightlies (CUDA 12.1.1)
+          exec: linux-x86_64-gpu
           label: nightly
+          raft: "ON"
+          cuda: "12.1.1"
+          cuda_archs: "70-real;72-real;75-real;80;86-real"
+          compiler_version: "11.2"
       - build_conda:
-          name: OSX x86_64 nightlies
-          exec: macosx-x86_64-cpu
+          name: Windows x86_64 nightlies
+          exec: windows-x86_64-cpu
           label: nightly
       - build_conda:
           name: OSX arm64 nightlies

.gitignore

@@ -17,3 +17,4 @@
 /tests/test
 /tests/gtest/
 faiss/python/swigfaiss_avx2.swig
+faiss/python/swigfaiss_avx512.swig