433bb68f2f
Added next few Oracle params
2020-01-09 03:23:06 -05:00
e1da577ad0
Fixed formatting
2020-01-08 15:29:31 -05:00
9393fb7aca
Fixed bugs and improved logging in config recommendation tasks/views
2020-01-08 15:29:31 -05:00
8cec62160f
Create/format/save the recommended config for the latest result before the on_success method.
2020-01-08 15:29:31 -05:00
8001b658c9
Added management function similar to 'startcelery' except that it auto-reloads modified website files.
2020-01-07 04:04:06 -05:00
9fced864fe
Updated Django settings: removed unused paths, fixed incorrect paths to media/template dirs, increased size of rotating log files from 50KB to 2MB and the number of backup files.
2020-01-03 07:32:28 -05:00
16acc35f54
Fix style
2019-12-21 06:47:48 +01:00
d007d932b5
Added target objective to comparison
2019-12-21 06:11:29 +01:00
e55aef5483
fix style
2019-12-20 19:08:16 +01:00
13e92bed6a
fix style
2019-12-20 19:08:16 +01:00
9633b0e99c
improve dnn
2019-12-20 19:08:16 +01:00
3a722df5e2
add missing columns in metrics
2019-12-17 10:02:49 +01:00
0aaa0d37f0
fix target objective for restart failure
2019-12-17 10:02:18 +01:00
f0deb63cdf
reset graph in DNN
2019-12-17 10:01:31 +01:00
c76c8e7bfb
fix multithread issue in DNN
2019-12-17 10:01:31 +01:00
2a7bc6145f
Fixed db time target objective
2019-12-16 20:47:24 +01:00
ade504dcc4
fix bug
2019-12-15 23:09:29 +01:00
df86a9f3bc
fix website tests
2019-12-15 23:09:29 +01:00
1ebcbf383e
regenerate website test fixture
2019-12-15 23:09:29 +01:00
477c8e7682
revert a LOG
2019-12-15 23:09:29 +01:00
7cc0c40d92
isolate workloads of different projects
2019-12-15 23:09:29 +01:00
fc4cf0e18b
added new oracle knobs
2019-12-15 19:20:29 +01:00
38f44848af
disable knob identification
2019-12-15 18:54:03 +01:00
ddd97a0973
fix knob identification
2019-12-15 18:54:03 +01:00
cae807cf62
add comment
2019-12-15 18:54:03 +01:00
b2e530095e
clean up
2019-12-15 18:54:03 +01:00
d5f7ae65eb
disable dummy encoder in knob identification
2019-12-15 18:54:03 +01:00
a144069e6b
shuffle lhs samples
2019-12-11 19:18:35 +01:00
6d76b22e3b
support enum/boolean in GPR/DNN
2019-12-11 09:18:40 +01:00
4b37dec988
Fixed formatting
2019-12-11 00:40:06 +01:00
ffc88bfaae
Run ddpg++ by default
2019-12-11 00:40:06 +01:00
65f3a7b0d6
Finished adding support for Oracle enums
2019-12-11 00:40:06 +01:00
a3cc34ba8e
Updated oracle fixtures
2019-12-10 18:01:56 +01:00
42e1a4add3
Make constants editable
2019-12-10 18:01:56 +01:00
d1a9933808
fix bug
2019-12-10 17:54:54 +01:00
7ade03e3e5
fix bugs
2019-12-10 10:14:01 +01:00
642d7c3875
fix enum bug
2019-12-10 10:14:01 +01:00
a7724233d3
fixed formatting
2019-12-09 22:19:47 +01:00
b0cddcc289
Added 3 Oracle hidden knobs
2019-12-09 22:19:47 +01:00
02c40f5cea
Adjusted background task time
2019-12-05 20:46:02 -05:00
783b94cd4d
Speedup travis-ci tests
2019-12-05 20:46:02 -05:00
9e9c9c7510
Fixed bug when collecting session knobs; for LHS sessions, remove all pre-generated configs if the session knobs are modified
2019-12-05 20:46:02 -05:00
1250732add
resolve conflicts
2019-12-05 03:45:18 -05:00
5579d8d94f
restore CDBTune
2019-12-05 03:45:18 -05:00
1e17c78956
debug for new gpr
2019-12-05 01:50:41 -05:00
70f9f952d5
hyperparameter debug info for new gpr
2019-12-05 01:50:41 -05:00
f66543d965
add white kernel
2019-12-04 18:07:36 -05:00
edb1aba4fc
add comments
2019-12-04 18:07:36 -05:00
7b57eb98a2
add tests for new gpr models
2019-12-04 18:07:36 -05:00
7ebb2ea473
update old gpr model
2019-12-04 18:07:36 -05:00
fd7f111c26
remove white kernel in new GPR
2019-12-04 18:07:36 -05:00
5654d23637
fix sigma in new gpr model
2019-12-04 18:07:36 -05:00
7ee615a3f3
change default ridge to 1.0
2019-12-04 18:07:36 -05:00
5e51cf7025
fix old gpr model, add noise variance
2019-12-04 18:07:36 -05:00
7be5b89975
Changes: when we deploy our docker images we now also build/deploy the internal driver image; fixed .dockerignore; moved integration test data into the driver; fixed cast in oracle db_time target objective.
2019-12-04 17:27:05 -05:00
4e5db94697
move restart location; auto fix knob range
2019-12-03 21:28:13 -05:00
c739eb066e
Changes: removed docker centos image; updated expected value used in test in analysis/tests/ddpg_test.py to be less strict; run all tests on travis vm and docker containers; push docker images to docker-hub repo if on the master branch;
2019-12-03 18:33:04 -05:00
cd36ff9803
Fixed Formatting
2019-12-03 17:28:35 -05:00
6ab7b8ef4f
Bug fixes
2019-12-03 17:28:35 -05:00
13735c9b0b
fix bugs
2019-12-03 17:28:35 -05:00
0dff040012
Initial code to send/manage DB error
2019-12-03 17:28:35 -05:00
e80639c859
Fixed formatting
2019-11-26 15:25:06 -05:00
9994a22f16
Updated management commands
2019-11-26 15:25:06 -05:00
4ab2fdfd52
Updated knob tunability getter
2019-11-26 15:25:06 -05:00
b211ad455d
Fixed bug caused by stale value of latest result in views.give_result()
2019-11-26 05:15:44 -05:00
e47508d20b
change knob catelog when changing session knob tunability
2019-11-25 17:55:15 -05:00
73cd8489f7
fix variable name
2019-11-25 13:22:54 -05:00
69dacaf193
make simulation interface consistent with website
2019-11-25 13:22:54 -05:00
b149684d77
add GPFLOW to website
2019-11-25 13:22:54 -05:00
dceee6e0ba
support Oracle flash back; auto-fix metric type; compile ojdbc
2019-11-25 13:17:53 -05:00
5555ead3a3
Fixed small bug in parser.
2019-11-22 15:12:54 -05:00
27f9cb55fd
fix bug in simulation.py
2019-11-22 20:04:04 +00:00
f34e879cb1
python style
2019-11-21 00:02:11 -05:00
f0c6d7ef1f
accelerate simulation; scale beta
2019-11-21 00:02:11 -05:00
9055516737
Updating Oracle knobs/metrics
2019-11-18 13:04:14 -05:00
a99cf72c6c
Updated oracle tunable knobs
2019-11-18 13:04:14 -05:00
a5cbb50521
fixed formatting
2019-11-18 13:04:14 -05:00
cf19380107
Updated the knob/metric output samples for each of the Oracle versions we support since my fix changed all of them.
2019-11-18 13:04:14 -05:00
e1b29eaafa
Added support for Oracle v12.1c.
2019-11-18 13:04:14 -05:00
d874ca6a8b
The Oracle collector now prepends the view name to each metric to avoid overwriting metrics from different views with the same names
2019-11-18 13:04:14 -05:00
6283186d76
Driver now works when the database system is local, remote, or on docker
2019-11-18 13:04:14 -05:00
21fce27291
improve ddpg
2019-11-18 10:51:22 -05:00
67a4a70c09
small fix
2019-11-08 05:49:19 -05:00
abbc589880
removed invalid field
2019-11-08 05:49:19 -05:00
4171a662f5
Added backdoor functions for creating/editing/deleting users/projects/sessions in the website and driver.
2019-11-08 05:49:19 -05:00
b261cf2139
Added migration for LHS
2019-11-06 21:05:33 -05:00
ee53d0cead
remove contextual gpr
2019-11-04 03:15:33 -05:00
e86fce59d9
import Dana's new gpr model
2019-11-04 03:15:33 -05:00
8bb3de85e7
update integration tests
2019-11-03 15:10:32 -05:00
576bbb69bf
fix style
2019-11-03 15:10:32 -05:00
2fa97149d5
add integration test
2019-11-03 15:10:32 -05:00
9f71d1c8de
improve simulator and ddpg
2019-10-28 20:55:06 -04:00
5431154784
add dnn and gpr to simulation
2019-10-24 22:29:49 -04:00
5c855faa3e
Testing gitlint
2019-10-24 00:54:38 -04:00
9f3fe29c27
added the compression option to the main settings file
2019-10-23 22:41:55 -04:00
40024b4d3c
Added an option to enable compression on MySQL tables (disabled by default)
2019-10-23 22:41:55 -04:00
ec896d01f6
Updated MySQL-specific migration that enables compression to be non-atomic
2019-10-23 22:41:55 -04:00
abe439a8e1
Bump django from 1.10.1 to 1.11.23 in /server/website
...
Bumps [django](https://github.com/django/django ) from 1.10.1 to 1.11.23.
- [Release notes](https://github.com/django/django/releases )
- [Commits](https://github.com/django/django/compare/1.10.1...1.11.23 )
Signed-off-by: dependabot[bot] <support@github.com>
2019-10-23 22:41:55 -04:00
21f4f40b88
simplify ddpg
2019-10-23 19:54:21 -04:00
336221d886
Bump requests from 2.18.4 to 2.20.0 in /server/website
...
Bumps [requests](https://github.com/requests/requests ) from 2.18.4 to 2.20.0.
- [Release notes](https://github.com/requests/requests/releases )
- [Changelog](https://github.com/psf/requests/blob/master/HISTORY.md )
- [Commits](https://github.com/requests/requests/compare/v2.18.4...v2.20.0 )
Signed-off-by: dependabot[bot] <support@github.com>
2019-10-22 21:40:52 -04:00
777c42c979
Bumped numpy from 1.13.1 to 1.14.0 to satisfy TF 1.12.2 dependency. Updated grep commands in validator to also be compatible with BSD.
2019-10-22 19:45:39 -04:00
68fedf765d
Bump tensorflow from 1.10 to 1.12.2 in /server/website
...
Bumps [tensorflow](https://github.com/tensorflow/tensorflow ) from 1.10 to 1.12.2.
- [Release notes](https://github.com/tensorflow/tensorflow/releases )
- [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md )
- [Commits](https://github.com/tensorflow/tensorflow/compare/v1.10.0...v1.12.2 )
Signed-off-by: dependabot[bot] <support@github.com>
2019-10-22 19:45:39 -04:00
826137bd3b
add simulation to white-list
2019-10-22 19:42:58 -04:00
d71c131e5b
work around a pylint & astroid bug
2019-10-22 19:42:58 -04:00
794418d29f
simulate a DB tuning environment to test DDPG performance
2019-10-22 19:42:58 -04:00
b215b156a4
Moved LHS to Server
2019-10-21 17:35:28 -04:00
a2234d08cc
set default minval of knob according to hardware
2019-10-15 21:01:53 -04:00
e4a97d060d
add command resetwebsite
2019-10-15 15:01:55 -04:00
fb7ee5c6a3
dump and load session knob tunability
2019-10-15 15:01:55 -04:00
ab17aafac4
modify ddpg test
...
change the expectation values in dnn test
2019-10-15 10:41:38 -04:00
dcb3152ec5
Added valid target objectives to the session create/edit form
2019-10-14 19:08:43 -04:00
01b4ec3f53
Support adding custom target objectives to the website/db/*/target_objective.py modules
2019-10-14 19:08:43 -04:00
f68c23e975
Added outer db module and moved parser code into it
2019-10-14 19:08:43 -04:00
a3833d83b1
normalize reward; square reward
2019-10-14 17:52:04 -04:00
090387a176
change the expected value in ddpg test
2019-10-14 17:52:04 -04:00
f071a0e62c
modify the hyper parameter and reward of ddpg
2019-10-14 17:52:04 -04:00
a5f393cdc1
test for dnn
2019-10-09 00:19:56 -04:00
0da1b724cc
Added postgresql client package to Dockerfiles and psycopg2-binary to requirements.txt, removed unused matplotlib code/package from analysis/cluster.py and requirements.txt, replaced commands in docker start.sh file with new management commands.
2019-10-08 09:32:25 -04:00
e8f28ebef0
Added another function that converts bytes/ms to human-readable values without loss of resolution.
2019-10-08 09:32:25 -04:00
3d70990ff1
The calculation of maxval for session knobs that use memory/storage resources is now set to 80% of the available resource (versus 100%)
2019-10-08 09:32:25 -04:00
c568c09c00
Enforce unique Project.name for same user and unique Session.name for same user and project
2019-10-08 09:32:25 -04:00
40c75de3ce
Added additional Oracle views to metric collection (v, v).
2019-10-08 09:32:25 -04:00
162dc48c53
Added option to dump debug info in 3 places: (1) the session view, (2) as a command in manage.py, and (3) as a fab command in the driver.
2019-10-08 09:32:25 -04:00
309d327a44
use assertAlmostEqual instead of assertEqual
2019-10-08 08:58:51 -04:00
f08204b9d6
Add testing for ddpg model
2019-10-08 08:58:51 -04:00
876d975713
improve UI
2019-10-08 08:29:10 -04:00
c5702e9159
tweak alignment in UI
2019-10-08 08:29:10 -04:00
924cccc459
update navbar in UI
2019-10-08 08:29:10 -04:00
63b7dd9a77
modify session page UI
2019-10-08 08:29:10 -04:00
e676433ece
Changes:
...
- fixed bug in validator causing it to validate files in excluded directories and updated the pre-commit hook
- hardware uniqueness is now based on the cpu, memory, storage, and storage_type fields, fixes #230
- initial settings of a minval/maxval for a session knob are calculated based on its resource type and the user's hardware
- squashed migrations
2019-10-03 21:31:27 -04:00
c14a337695
Removed obsolete methods from parser and fields from Session model
2019-10-03 21:31:27 -04:00
b3c42a81fb
Added django-db-logger for storing log messages in the database, removed random *.sh scripts from website code then extended manage.py with same commands
2019-10-03 21:31:27 -04:00
6b8aeb6043
fixed expected status_codes in tests
2019-10-03 21:31:27 -04:00
390e58b24d
added some manage.py extensions
2019-10-03 21:31:27 -04:00
e5edd2a7f4
Fixed celery task logging and added filehandler
2019-10-03 21:31:27 -04:00
863d6619ba
Fixed incorrect parsing behavior for float-type metrics
2019-10-03 21:31:27 -04:00
d1c13de42f
Consolidated driver scripts and added status codes to the query_and_get and new_result responses
2019-10-03 21:31:27 -04:00
8743c0c2b1
support oracle v12 in ottertune server
2019-10-03 18:58:30 -04:00
494b23678e
use algorithm tpye
2019-09-30 01:34:30 -04:00
e9f503ef3e
change to algoroithm tpyes
2019-09-30 01:34:30 -04:00
25d0838376
save dnn model into database
2019-09-30 01:34:30 -04:00
c37ef9c072
move dnn parameters to constant file
2019-09-30 01:34:30 -04:00
82a7d859c2
choose algorithm based on option
2019-09-30 01:34:30 -04:00
be955cc812
fix typos
2019-09-30 01:34:30 -04:00
e1b97bca9b
upgrade tensorflow
2019-09-30 01:34:30 -04:00
f716ade292
add noise exploration
2019-09-30 01:34:30 -04:00
11cfe5afc1
add deep learning model
2019-09-30 01:34:30 -04:00
d7d7f9111f
Added new DDPG columns to migrations
2019-09-27 18:18:32 -04:00
2c619e017f
fix python style
2019-09-27 00:36:21 -04:00
5ad0e8c44e
delete unused algorithm 3
2019-09-27 00:36:21 -04:00
dvanaken
arifiorino
bohanjason
Bohan Zhang
yangdsh
dependabot[bot]
DongshengYang
bohanjason
Dongsheng Yang