Skip to content
Snippets Groups Projects
  1. Aug 31, 2015
  2. Aug 28, 2015
    • Andres Noetzli's avatar
      Fix deadlock in WAL sync · 297241e0
      Andres Noetzli authored
      Summary:
      MarkLogsSynced() was doing `logs_.erase(it++);`. The standard is saying:
      
      ```
      all iterators and references are invalidated, unless the erased members are at an end (front or back) of the deque (in which case only iterators and references to the erased members are invalidated)
      ```
      
      Because `it` is an iterator to the first element of the container, it is
      invalidated, only one iteration is executed and `log.getting_synced = false;`
      is not being done, so `while (logs_.front().getting_synced)` in `WriteImpl()`
      is not terminating.
      
      Test Plan: make db_bench && ./db_bench --benchmarks=fillsync
      
      Reviewers: igor, rven, IslamAbdelRahman, anthony, kradhakrishnan, yhchiang, sdong, tnovak
      
      Reviewed By: tnovak
      
      Subscribers: kolmike, dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D45807
      297241e0
  3. Aug 06, 2015
  4. Aug 05, 2015
    • sdong's avatar
      Add util/delete_scheduler_impl.cc to CMakeLists.txt · 48e6e9aa
      sdong authored
      Summary: util/delete_scheduler_impl.cc is missing CMakeLists.txt, which fails Windows build. Add it.
      
      Test Plan: Build in both of Windows and Linux
      
      Reviewers: anthony, rven, yhchiang, IslamAbdelRahman
      
      Reviewed By: IslamAbdelRahman
      
      Subscribers: leveldb, dhruba
      
      Differential Revision: https://reviews.facebook.net/D43641
      48e6e9aa
    • sdong's avatar
      Add Statistics.getHistogramString() to print more detailed outputs of a histogram · c7742452
      sdong authored
      Summary:
      Provide a way for users to know more detailed ditribution of a histogram metrics. Example outputs:
      
      Manually add statement
        fprintf(stdout, "%s\n", dbstats->getHistogramString(SST_READ_MICROS).c_str());
      Will print out something like:
      
      Count: 989151  Average: 1.7659  StdDev: 1.52
      Min: 0.0000  Median: 1.2071  Max: 860.0000
      Percentiles: P50: 1.21 P75: 1.70 P99: 5.12 P99.9: 13.67 P99.99: 21.70
      ------------------------------------------------------
      [       0,       1 )   390839  39.513%  39.513% ########
      [       1,       2 )   500918  50.641%  90.154% ##########
      [       2,       3 )    79358   8.023%  98.177% ##
      [       3,       4 )     6297   0.637%  98.813%
      [       4,       5 )     1712   0.173%  98.986%
      [       5,       6 )     1134   0.115%  99.101%
      [       6,       7 )     1222   0.124%  99.224%
      [       7,       8 )     1529   0.155%  99.379%
      [       8,       9 )     1264   0.128%  99.507%
      [       9,      10 )      988   0.100%  99.607%
      [      10,      12 )     1378   0.139%  99.746%
      [      12,      14 )     1828   0.185%  99.931%
      [      14,      16 )      410   0.041%  99.972%
      [      16,      18 )       72   0.007%  99.980%
      [      18,      20 )       67   0.007%  99.986%
      [      20,      25 )      106   0.011%  99.997%
      [      25,      30 )       24   0.002%  99.999%
      [      30,      35 )        1   0.000% 100.000%
      [     250,     300 )        2   0.000% 100.000%
      [     300,     350 )        1   0.000% 100.000%
      [     800,     900 )        1   0.000% 100.000%
      
      Test Plan: Manually add a print in db_bench and make sure it prints out as expected. Will add some codes to cover the function
      
      Subscribers: leveldb, dhruba
      
      Differential Revision: https://reviews.facebook.net/D43611
      c7742452
    • Islam AbdelRahman's avatar
      Make DeleteScheduler tests more reliable · 29b028b0
      Islam AbdelRahman authored
      Summary: Update DeleteScheduler tests so that they verify the used penalties for waiting instead of measuring the time spent which is not reliable
      
      Test Plan:
      make -j64 delete_scheduler_test && ./delete_scheduler_test
      COMPILE_WITH_TSAN=1 make -j64 delete_scheduler_test && ./delete_scheduler_test
      COMPILE_WITH_ASAN=1 make -j64 delete_scheduler_test && ./delete_scheduler_test
      
      make -j64 db_test && ./db_test --gtest_filter="DBTest.RateLimitedDelete:DBTest.DeleteSchedulerMultipleDBPaths"
      COMPILE_WITH_TSAN=1 make -j64 db_test && ./db_test --gtest_filter="DBTest.RateLimitedDelete:DBTest.DeleteSchedulerMultipleDBPaths"
      COMPILE_WITH_ASAN=1 make -j64 db_test && ./db_test --gtest_filter="DBTest.RateLimitedDelete:DBTest.DeleteSchedulerMultipleDBPaths"
      
      Reviewers: yhchiang, sdong
      
      Reviewed By: sdong
      
      Subscribers: dhruba
      
      Differential Revision: https://reviews.facebook.net/D43635
      29b028b0
    • sdong's avatar
      valgrind_check to exit on test failures · fca88f8e
      sdong authored
      Summary: Currently, valgrind_check doesn't fail on test failures, which creates confusion. valgrind_check should fail if test fails.
      
      Test Plan: Manually change tests to return test failure or cause memory leak and see valgrind_check has the correct behavior.
      
      Reviewers: anthony, yhchiang, IslamAbdelRahman, igor, kradhakrishnan, rven
      
      Reviewed By: rven
      
      Subscribers: leveldb, dhruba
      
      Differential Revision: https://reviews.facebook.net/D43629
      fca88f8e
    • Poornima Chozhiyath Raman's avatar
      Fix build failure · 7d364d0d
      Poornima Chozhiyath Raman authored
      Summary: fix the build failure
      
      Test Plan: make all
      
      Reviewers: sdong, IslamAbdelRahman
      
      Reviewed By: IslamAbdelRahman
      
      Subscribers: dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D43623
      7d364d0d
    • Poornima Chozhiyath Raman's avatar
      Add function 'GetInfoLogList()' · 960d936e
      Poornima Chozhiyath Raman authored
      Summary: The list of info log files of a db can be obtained using the new function.
      
      Test Plan: New test in db_test.cc passed.
      
      Reviewers: yhchiang, IslamAbdelRahman, sdong
      
      Reviewed By: sdong
      
      Subscribers: IslamAbdelRahman, leveldb, dhruba
      
      Differential Revision: https://reviews.facebook.net/D41715
      960d936e
    • sdong's avatar
      Add two unit tests for SyncWAL() · 7ccd1c80
      sdong authored
      Summary:
      Add two unit tests for SyncWAL(). One makes sure SyncWAL() doesn't block writes in the other thread. Another one makes sure SyncWAL() doesn't wait ongoing writes to finish before being executed.
      
      Create a new test file db_wal_test and move two WAL related tests from db_test to here.
      
      Test Plan: Run the new tests
      
      Reviewers: IslamAbdelRahman, rven, kradhakrishnan, kolmike, tnovak, yhchiang
      
      Reviewed By: yhchiang
      
      Subscribers: leveldb, dhruba
      
      Differential Revision: https://reviews.facebook.net/D43605
      7ccd1c80
    • sdong's avatar
      Add statistic histogram "rocksdb.sst.read.micros" · 3ae386ea
      sdong authored
      Summary: Measure read latency histogram and put in statistics. Compaction inputs are excluded from it when possible (unfortunately usually no possible as we usually take table reader from table cache.
      
      Test Plan:
      Run db_bench and it shows the stats, like:
      
      rocksdb.sst.read.micros statistics Percentiles :=> 50 : 1.238522 95 : 2.529740 99 : 3.912180
      
      Reviewers: kradhakrishnan, rven, anthony, IslamAbdelRahman, MarkCallaghan, yhchiang
      
      Reviewed By: yhchiang
      
      Subscribers: leveldb, dhruba
      
      Differential Revision: https://reviews.facebook.net/D43275
      3ae386ea
    • sdong's avatar
      "make commit-prereq" should clean up rocksjava properly · 8ecb51a7
      sdong authored
      Summary: "make commit-prereq" fails to clean up java, which can cause rocksjava failure.
      
      Test Plan: Run commit-prepreq
      
      Reviewers: IslamAbdelRahman, rven, kradhakrishnan, yhchiang
      
      Reviewed By: yhchiang
      
      Subscribers: leveldb, dhruba
      
      Differential Revision: https://reviews.facebook.net/D43575
      8ecb51a7
    • Islam AbdelRahman's avatar
      Enable DBTest.FlushSchedule under TSAN · 9aec75fb
      Islam AbdelRahman authored
      Summary: This patch will fix the false positive of DBTest.FlushSchedule under TSAN, we dont need to disable this test
      
      Test Plan: COMPILE_WITH_TSAN=1 make -j64 db_test && ./db_test --gtest_filter="DBTest.FlushSchedule"
      
      Reviewers: yhchiang, sdong
      
      Reviewed By: sdong
      
      Subscribers: dhruba
      
      Differential Revision: https://reviews.facebook.net/D43599
      9aec75fb
    • Islam AbdelRahman's avatar
      Fix TSAN for delete_scheduler_test · bd2fc5f5
      Islam AbdelRahman authored
      Summary: Fixing TSAN false positive and relaxing the conditions when we are running under TSAN
      
      Test Plan: COMPILE_WITH_TSAN=1 make -j64 delete_scheduler_test && ./delete_scheduler_test
      
      Reviewers: yhchiang, sdong
      
      Reviewed By: sdong
      
      Subscribers: dhruba
      
      Differential Revision: https://reviews.facebook.net/D43593
      bd2fc5f5
    • sdong's avatar
      Fix misplaced position for reversing iterator direction while current key is a merge · 8e01bd11
      sdong authored
      Summary:
      While doing forward iterating, if current key is merge, internal iterator position is placed to the next key. If Prev() is called now, needs to do extra Prev() to recover the location.
      This is second attempt of fixing after reverting ec70fea4. This time shrink the fix to only merge key is the current key and avoid the reseeking logic for max_iterating skipping
      
      Test Plan: enable the two disabled tests and make sure they pass
      
      Reviewers: rven, IslamAbdelRahman, kradhakrishnan, tnovak, yhchiang
      
      Reviewed By: yhchiang
      
      Subscribers: leveldb, dhruba
      
      Differential Revision: https://reviews.facebook.net/D43557
      8e01bd11
    • Andres Notzli's avatar
      Removing duplicate code · c4650710
      Andres Notzli authored
      Summary:
      While working on https://reviews.facebook.net/D43179 , I found
      duplicate code in the tests. This patch removes it.
      
      Test Plan: make clean all check
      
      Reviewers: igor, sdong, rven, anthony, yhchiang
      
      Reviewed By: yhchiang
      
      Subscribers: dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D43263
      c4650710
    • Mike Kolupaev's avatar
      [wal changes 3/3] method in DB to sync WAL without blocking writers · e06cf1a0
      Mike Kolupaev authored
      Summary:
      Subj. We really need this feature.
      
      Previous diff D40899 has most of the changes to make this possible, this diff just adds the method.
      
      Test Plan: `make check`, the new test fails without this diff; ran with ASAN, TSAN and valgrind.
      
      Reviewers: igor, rven, IslamAbdelRahman, anthony, kradhakrishnan, tnovak, yhchiang, sdong
      
      Reviewed By: sdong
      
      Subscribers: MarkCallaghan, maykov, hermanlee4, yoshinorim, tnovak, dhruba
      
      Differential Revision: https://reviews.facebook.net/D40905
      e06cf1a0
    • Ari Ekmekji's avatar
      Update Tests To Enable Subcompactions · 5dc3e688
      Ari Ekmekji authored
      Summary:
      Updated DBTest DBCompactionTest and CompactionJobStatsTest
      to run compaction-related tests once with subcompactions enabled and
      once disabled using the TEST_P test type in the Google Test suite.
      
      Test Plan: ./db_test  ./db_compaction-test  ./compaction_job_stats_test
      
      Reviewers: sdong, igor, anthony, yhchiang
      
      Reviewed By: yhchiang
      
      Subscribers: dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D43443
      5dc3e688
  5. Aug 04, 2015
Loading