emGee Software Solutions Custom Database Applications

Share this

Web Technologies

Windows Tools for MySQL DBAs: Basic Minidump Analysis

Planet MySQL - Sun, 03/25/2018 - 09:17
"To a man with a hammer, everything looks like a nail."Even though I had written many posts explaining the use of gdb for various MySQL-related tasks, I have to use other OS level troubleshooting tools from time to time. Moreover, as MySQL and MariaDB are still supported and used under Microsoft Windows in production by customers I have to serve them there, and use Windows-specific tools sometimes. So, I decided to start a series of posts (that I promised to my great colleague Vladislav Vaintroub (a.k.a Wlad) who helped me a lot over years and actually switched my attention from Performance Schema towards debuggers) about different Windows tools for MySQL DBAs (and support engineers).

Developers (and maybe even power users) on Windows probably know all I plan to describe and way more, by heart, but for me many things were not obvious and took some time to search, try or even ask for some advises... So, this series of posts is going to be useful at least for me (and mostly UNIX users, like me), as a source of hints and links that may save me some time and efforts in the future.

In this first post I plan to describe basic installation of "Debugging Tools for Windows" and use of cdb command line debugger to analyze minidumps (that one gets on Windows upon crashes when core-file option is added to my.ini and may get for hanging mysqld.exe process with minimal efforts using different tools) and get backtraces and few other details from them. I also plan to show simple command lines to share with DBAs and users whom you help, that allow to get useful details (more or less full backtraces, crash analysis, OS details etc) for further troubleshooting when/if dumps can not or should not be shared.
--- I have to confess: I use Microsoft Windows on desktops and laptops. I started from Windows 3.0 back in 1992 and ended with Windows 10 on my wife's laptop. I use Windows even for work. Today 2 of my 4 machines used for work-related tasks run Windows (64-bit XP on old Dell box I've got from MySQL AB back in 2005 and 64-bit Windows 7 on this Acer netbook). At the same time, most of work I have to do since 1992 is related to UNIX of all kinds (from Xenix and SCO OpenDesktop that I connected to from VT220 terminal in at my first job after the university, to recent Linux versions used by customers in production, my Fedora 27 box and Ubuntu 14.04 netbook used as build, Docker, VirtualBox, testing, benchmarking etc servers). I had never become a real powerful user of Windows (no really complex .bat files, PowerShell programming or even Basic macros in Word, domains, shadow copy services usage for backups, nothing fancy). But on UNIX I had to master shell, vi :), some Perl and a lot of command line tools.

I had to do some software development on Windows till 2005, built MySQL on Windows sometimes up to 2012 when I joined Percona (that had nothing to do with Windows at all), so I have old version of Visual Studio, some older WinDbg and other debugging tools here and there, but had not used them more than once a year, until recently... Last time I attached WinDbg to anything MySQL-related it was MariaDB 10.1.13, during some troubleshooting related to MDEV-10191.

Suddenly in March I've got issues from customers related to hanging upon startup/InnoDB recovery and under load, and crashing while using some (somewhat exotic) storage engine, all these - on modern versions of Microsoft Windows, in production. I had no other option but to get and study backtraces (of all threads or crashing threads) and check source code. It would be so easy to get them on Linux (just ask them to install gdb , attach it to hanging mysqld process or point out to the mysqld binary and core, and get the output of thread apply all backtrace, minor details aside). But how to do this on Winsdows, in command line if possible (as I hate to share screenshots and write long explanations on where to click and what to copy/paste)? I had to check in WinDbg, get some failures because of my outdated and incomplete environment (while customer with proper environment provided useful outputs anyway), then, eventually, asked Wlad for some help. Eventually I was able to make some progress.

To be ready to do this again next time with confidence, proper test environment and without wasting anybody else's time, I decided to repeat some of these efforts in clean environment and make notes, that I am going to share in this series of blog posts. Today I'll concentrate on installing current "Debugging Tools for Windows" and using cdb from them to process minidumps.

1. Installing "Debugging Tools for Windows"There is a nice, easy to find document from Microsoft on how to get cdb and other debugging tools for Windows. For recent versions you just have to download Windows 10 SDK and then install these tools (and everything else you may need) from it. Proceed to this page, read the details, click on "Download .EXE" to get winsdksetup.exe , start it and select "Debugging Tools for Windows" when requested to select the features. Eventually you'll get some 416+ MB downloaded and installed by default in C:\Program Files (x86)\Windows Kits\10\Debuggers\. (on default 64-bit Windows installation with C: as system disk). Quick check shows I have everything I need:
C:\Program Files (x86)\Windows Kits\10\Debuggers\x64>dir
...
11/10/2017  11:55 PM           154,936 cdb.exe
...
11/10/2017  11:55 PM           576,312 windbg.exe
...Here is the list of most useful cdb options for the next step:
C:\Program Files (x86)\Windows Kits\10\Debuggers\x64>cdb /?
cdb version 10.0.16299.91
usage: cdb [options]

Options:

  <command-line> command to run under the debugger
  -? displays command line help text
...
  -i <ImagePath> specifies the location of the executables that generated the
                 fault (see _NT_EXECUTABLE_IMAGE_PATH)
...
  -lines requests that line number information be used if present
...
  -logo <logfile> opens a new log file
...
  -p <pid> specifies the decimal process ID to attach to
...
  -pv specifies that any attach should be noninvasive
...
  -y <SymbolsPath> specifies the symbol search path (see _NT_SYMBOL_PATH)
  -z <CrashDmpFile> specifies the name of a crash dump file to debug
...
Environment Variables:

    _NT_SYMBOL_PATH=[Drive:][Path]
        Specify symbol image path.
...
Control Keys:

     <Ctrl-B><Enter> Quit debugger
...Remember Crtl-B key combination as a way to quit from cdb. I looked as funny as the beginner vi user few times, clicking on everything to get out of that tool...

2. Basic Use of cdb to Process MinidumpLet's assume you've got mysqld.dmp minidump file (a kind of "core" file on UNIX, but better, at least smaller usually) created during some crash. Depending on binaries used, you may need to make sure you have .PDB files in some directory, for the mysqld.exe binary and all .dll files for plugins/extra storage engines used, in some directory. Default path to .PDB files is defined by the _NT_SYMBOL_PATH environment variable and may include multiple directories ad URLs.

Initially I've got advice to set this environment variable as follows:
set _NT_SYMBOL_PATH=srv*c:\symbols*http://msdl.microsoft.com/download/symbolsThis assumes that I have a collection of .PDB files in c:\symbols on some locally available server and rely on Microsoft's symbols server for the rest. For anything missing we can always add -y option to point to some directory with additional .PDB files. Note that MariaDB provides .pdb files along with .exe in .msi installer, not only in .zip file with binaries.

So, if your mysqld.dmp file is located in h:\, mysqld.exe for the same version as generated that minidump is located in p:\software and all related .dll files and .pdb files for them all are also there, the command to get basic details about the crash in file h:\out.txt would be the following:
cdb -z h:\mysqld.dmp -i p:\software -y p:\software -logo h:\out.txt -c "!sym prompts;.reload;.ecxr;q"You can click on every option underlined above to get details. It produces output like this:
C:\Program Files (x86)\Windows Kits\10\Debuggers\x64>cdb -z h:\mysqld.dmp -i p:\
software -y p:\software -logo h:\out.txt -c "!sym prompts;.reload;.ecxr;q"


Microsoft (R) Windows Debugger Version 10.0.16299.91 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.


Loading Dump File [h:\mysqld.dmp]
User Mini Dump File: Only registers, stack and portions of memory are available


************* Path validation summary **************
Response                         Time (ms)     Location
OK                                             p:\software

************* Path validation summary **************
Response                         Time (ms)     Location
OK                                             p:\software
Deferred                                       srv*c:\symbols*http://msdl.micros
oft.com/download/symbols
Symbol search path is: p:\software;srv*c:\symbols*http://msdl.microsoft.com/down
load/symbols
Executable search path is: p:\software
Windows 10 Version 14393 MP (4 procs) Free x64
Product: Server, suite: TerminalServer SingleUserTS
10.0.14393.206 (rs1_release.160915-0644)
Machine Name:
Debug session time: ...
System Uptime: not available
Process Uptime: 0 days X:YY:ZZ.000
............................................
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(1658.fd0): Access violation - code c0000005 (first/second chance not available)

ntdll!NtGetContextThread+0x14:
00007fff`804a7d84 c3              ret
0:053> cdb: Reading initial command '!sym prompts;.reload;.ecxr;q'
quiet mode - symbol prompts on
............................................
rax=0000000000000000 rbx=0000000000000000 rcx=0000000000000006
rdx=000001cf0ac2e118 rsi=000001cf0abeeef8 rdi=000001cf0ac2e118
rip=00007fff5f313b0d rsp=000000653804e2b0 rbp=000001cf165c9cc8
 r8=0000000000000000  r9=00007fff5f384448 r10=000000653804ef70
r11=000000653804eb28 r12=0000000000000000 r13=000001cf0ab49d48
r14=0000000000000000 r15=000001cf0b083028
iopl=0         nv up ei pl zr na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
ha_spider!spider_db_connect+0xdd:
00007fff`5f313b0d 8b1498          mov     edx,dword ptr [rax+rbx*4] ds:00000000`
00000000=????????
quit:

C:\Program Files (x86)\Windows Kits\10\Debuggers\x64>that also goes to the file pointed out by the -logo option. Here we have some weird crash in Spider engine of MariaDB that  is not a topic of current post.

If you think the crash is related to some activity of other threads, you can get all unique stack dumps with the following options:
cdb -lines -z h:\mysqld.dmp -i p:\software -y p:\software -logo h:\out.txt -c "!sym prompts;.reload;!uniqstack -p;q" This is how the backtrace of slave SQL thread may look like, note files with line numbers for each frame (-lines option): . 44  Id: 1658.1584 Suspend: 0 Teb: 00000065`32185000 Unfrozen
      Priority: 0  Priority class: 32
Child-SP          RetAddr           Call Site
00000065`353fed08 00007fff`8046d119 ntdll!NtWaitForAlertByThreadId+0x14
00000065`353fed10 00007fff`7cbd8d78 ntdll!RtlSleepConditionVariableCS+0xc9
00000065`353fed80 00007ff6`2d7d62e7 KERNELBASE!SleepConditionVariableCS+0x28
00000065`353fedb0 00007ff6`2d446c8e mysqld!pthread_cond_timedwait(struct _RTL_CO
NDITION_VARIABLE * cond = 0x000001ce`66805688, struct _RTL_CRITICAL_SECTION * mu
tex = 0x000001ce`668051b8, struct timespec * abstime = <Value unavailable error>
)+0x27 [d:\winx64-packages\build\src\mysys\my_wincond.c @ 85]
(Inline Function) --------`-------- mysqld!inline_mysql_cond_wait+0x61 [d:\winx6
4-packages\build\src\include\mysql\psi\mysql_thread.h @ 1149]
00000065`353fede0 00007ff6`2d4b1718 mysqld!MYSQL_BIN_LOG::wait_for_update_relay_
log(class THD * thd = <Value unavailable error>)+0xce [d:\winx64-packages\build\
src\sql\log.cc @ 8055]
00000065`353fee90 00007ff6`2d4af03f mysqld!next_event(struct rpl_group_info * rg
i = 0x000001ce`667fe560, unsigned int64 * event_size = 0x00000065`353ff008)+0x2b
8 [d:\winx64-packages\build\src\sql\slave.cc @ 7148]
00000065`353fef60 00007ff6`2d4bb038 mysqld!exec_relay_log_event(class THD * thd
= 0x000001ce`6682ece8, class Relay_log_info * rli = 0x000001ce`66804d58, struct
rpl_group_info * serial_rgi = 0x000001ce`667fe560)+0x8f [d:\winx64-packages\buil
d\src\sql\slave.cc @ 3866
]
00000065`353ff000 00007ff6`2d7d35cb mysqld!handle_slave_sql(void * arg = 0x00000
1ce`66803430)+0xa28 [d:\winx64-packages\build\src\sql\slave.cc @ 5145]
00000065`353ff780 00007ff6`2d852d51 mysqld!pthread_start(void * p = <Value unava
ilable error>)+0x1b [d:\winx64-packages\build\src\mysys\my_winthread.c @ 62]
(Inline Function) --------`-------- mysqld!invoke_thread_procedure+0xe [d:\th\mi
nkernel\crts\ucrt\src\appcrt\startup\thread.cpp @ 91]
00000065`353ff7b0 00007fff`80338364 mysqld!thread_start<unsigned int (void * par
ameter = 0x00000000`00000000)+0x5d [d:\th\minkernel\crts\ucrt\src\appcrt\startup
\thread.cpp @ 115]
00000065`353ff7e0 00007fff`804670d1 kernel32!BaseThreadInitThunk+0x14
00000065`353ff810 00000000`00000000 ntdll!RtlUserThreadStart+0x21 For crash analysis usually !analyze command is also used: cdb -lines -z h:\mysqld.dmp -i p:\software -y p:\software -logo h:\out.txt -c "!sym prompts;.reload;!analyze -v;q" It may give some details about the exception happened: ... FAULTING_IP:
ha_spider!spider_db_connect+dd00007fff`5f313b0d 8b1498          mov     edx,dword ptr [rax+rbx*4]

EXCEPTION_RECORD:  (.exr -1)
ExceptionAddress: 00007fff5f313b0d (ha_spider!spider_db_connect+0x00000000000000
dd)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 0000000000000000
   Parameter[1]: 0000000000000000
Attempt to read from address 0000000000000000
DEFAULT_BUCKET_ID:  NULL_POINTER_READ

PROCESS_NAME:  mysqld.exe

ERROR_CODE: (NTSTATUS) 0xc0000005 - <Unable to get error code text> ... STACK_TEXT:
00000065`3804e2b0 00007fff`5f3132ad : 00000000`00000000 000001cf`1741ab68 000001
cf`0b083028 000001cf`0ac2e118 : ha_spider!spider_db_connect+0xdd
00000065`3804e330 00007fff`5f3117f8 : 000001ce`669107c8 000001cf`0ac2e118 000001
cf`1741ab68 00000000`00000001 : ha_spider!spider_db_conn_queue_action+0xad
00000065`3804ea20 00007fff`5f31a1ee : 00000000`00000000 000001cf`0abeeef8 000000
00`00000000 000001cd`c0b30000 : ha_spider!spider_db_before_query+0x108
00000065`3804eaa0 00007fff`5f31a0bf : 00000000`00000000 00000000`00000000 000000
65`3804ec70 00000000`00000038 : ha_spider!spider_db_set_names_internal+0x11e
00000065`3804eb30 00007fff`5f369b4e : 00000000`00000000 00000065`3804ec70 00007f
ff`5f387f08 00000000`00000000 : ha_spider!spider_db_set_names+0x3f
00000065`3804eb70 00007fff`5f32f5f1 : 00000000`00000001 00000065`00000000 41cfff
ff`00000001 00000000`00000001 : ha_spider!spider_mysql_handler::show_table_statu
s+0x15e
00000065`3804ece0 00007fff`5f3222e8 : 00000000`00000001 00000065`00000000 000000
00`5ab175cd 000001cf`0b0886f8 : ha_spider!spider_get_sts+0x201
00000065`3804edb0 00007ff6`2d7d35cb : 00000000`00000057 000001cf`0ab49d48 000000
00`00000000 00007fff`5f321c10 : ha_spider!spider_bg_sts_action+0x6d8
00000065`3804fa30 00007ff6`2d852d51 : 000001cf`17008fe0 000001cf`0aa3fef0 000000
00`00000000 00000000`00000000 : mysqld!pthread_start+0x1b
00000065`3804fa60 00007fff`80338364 : 00000000`00000000 00000000`00000000 000000
00`00000000 00000000`00000000 : mysqld!thread_start<unsigned int (__cdecl*)(void
 * __ptr64)>+0x5d
00000065`3804fa90 00007fff`804670d1 : 00000000`00000000 00000000`00000000 000000
00`00000000 00000000`00000000 : kernel32!BaseThreadInitThunk+0x14
00000065`3804fac0 00000000`00000000 : 00000000`00000000 00000000`00000000 000000
00`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x21 ... Finally (for this post), this is how we can get information about a crashing thread, including details about local variables (like full backtrace in gdb). We apply !for_each_frame extension and use dv to "display variable": cdb -z h:\mysqld.dmp -i p:\software -y p:\software -logo h:\out.txt -c "!sym prompts;.reload;.ecxr;!for_each_frame dv /t;q" The result will include details about each frame, parameters and local variables, like this: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
00 00000065`3804e2b0 00007fff`5f3132ad ha_spider!spider_db_connect+0xdd
struct st_spider_share * share = 0x000001cf`165c9cc8
struct st_spider_conn * conn = 0x000001cf`0ac2e118
int link_idx = 0n0
int error_num = <value unavailable>
class THD * thd = 0x000001cf`0abeeef8
int64 connect_retry_interval = <value unavailable>
int connect_retry_count = <value unavailable>
int64 tmp_time = <value unavailable>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
01 00000065`3804e330 00007fff`5f3117f8 ha_spider!spider_db_conn_queue_action+0xa
d
struct st_spider_conn * conn = 0x000001cf`0ac2e118
int error_num = 0n0
char [1532] sql_buf = char [1532] ""
class spider_string sql_str = class spider_string
class spider_db_result * result = <value unavailable>
struct st_spider_db_request_key request_key = struct st_spider_db_request_key
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
02 00000065`3804ea20 00007fff`5f31a1ee ha_spider!spider_db_before_query+0x108
struct st_spider_conn * conn = 0x000001cf`0ac2e118
int * need_mon = 0x000001cf`1741ab68
int error_num = 0n0
bool tmp_mta_conn_mutex_lock_already = true
class ha_spider * spider = <value unavailable>
bool tmp_mta_conn_mutex_unlock_later = <value unavailable>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
03 00000065`3804eaa0 00007fff`5f31a0bf ha_spider!spider_db_set_names_internal+0x
11e
struct st_spider_transaction * trx = 0x000001cf`0b083028
struct st_spider_share * share = 0x000001cf`0ab49d48
struct st_spider_conn * conn = 0x000001cf`0ac2e118
int all_link_idx = 0n0
int * need_mon = 0x000001cf`1741ab68
bool tmp_mta_conn_mutex_lock_already = true
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
04 00000065`3804eb30 00007fff`5f369b4e ha_spider!spider_db_set_names+0x3f
class ha_spider * spider = <value unavailable>
struct st_spider_conn * conn = <value unavailable>
int link_idx = <value unavailable>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
05 00000065`3804eb70 00007fff`5f32f5f1 ha_spider!spider_mysql_handler::show_tabl
e_status+0x15e
class spider_mysql_handler * this = 0x000001cf`0a15dd00
int link_idx = 0n0
int sts_mode = 0n1
unsigned int flag = 1
int error_num = 0n1
struct st_spider_share * share = 0x000001cf`0ab49d48
struct st_spider_conn * conn = 0x000001cf`0ac2e118
class spider_db_result * res = <value unavailable>
unsigned int64 auto_increment_value = 0
unsigned int pos = 0
struct st_spider_db_request_key request_key = struct st_spider_db_request_key
struct st_spider_db_request_key request_key = struct st_spider_db_request_key
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
06 00000065`3804ece0 00007fff`5f3222e8 ha_spider!spider_get_sts+0x201
struct st_spider_share * share = 0x000001cf`0ab49d48
int link_idx = 0n0
int64 tmp_time = 0n1521579469
class ha_spider * spider = 0x00000065`3804ef70
double sts_interval = 10
int sts_mode = 0n1
int sts_sync = 0n0
int sts_sync_level = 0n2
unsigned int flag = 0x18
int error_num = <value unavailable>
int get_type = 0n1
struct st_spider_patition_handler_share * partition_handler_share = <value unava
ilable>
double tmp_sts_interval = <value unavailable>
struct st_spider_share * tmp_share = <value unavailable>
int tmp_sts_sync = <value unavailable>
class ha_spider * tmp_spider = <value unavailable>
int roop_count = <value unavailable>
int tmp_sts_mode = <value unavailable>
class THD * thd = <value unavailable>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
07 00000065`3804edb0 00007ff6`2d7d35cb ha_spider!spider_bg_sts_action+0x6d8
void * arg = 0x000001cf`0ab49d48
int error_num = 0n0
class ha_spider spider = class ha_spider
unsigned int * conn_link_idx = 0x000001cf`1741ab78
unsigned char * conn_can_fo = 0x000001cf`1741ab80 "--- memory read error at addr
ess 0x000001cf`1741ab80 ---"
struct st_spider_conn ** conns = 0x000001cf`1741ab70
int * need_mons = 0x000001cf`1741ab68
int roop_count = 0n0
char ** conn_keys = 0x000001cf`1741ab88
class THD * thd = 0x000001cf`0abeeef8
class spider_db_handler ** dbton_hdl = 0x000001cf`1741ab90
struct st_spider_transaction * trx = 0x000001cf`0b083028
struct st_mysql_mutex spider_global_trx_mutex = <value unavailable>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
08 00000065`3804fa30 00007ff6`2d852d51 mysqld!pthread_start+0x1b
void * p = <value unavailable>
void * arg = 0x000001cf`0ab49d48
<function> * func = 0x00007fff`5f321c10
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
09 (Inline Function) --------`-------- mysqld!invoke_thread_procedure+0xe
void * context = 0x000001cf`17008fe0
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
0a 00000065`3804fa60 00007fff`80338364 mysqld!thread_start<unsigned int (__cdecl
*)(void * __ptr64)>+0x5d
void * parameter = 0x00000000`00000000
<function> * procedure = 0x00007ff6`2d7d35b0
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
0b 00000065`3804fa90 00007fff`804670d1 kernel32!BaseThreadInitThunk+0x14
Unable to enumerate locals, Win32 error 0n87
Private symbols (symbols.pri) are required for locals.
Type ".hh dbgerr005" for details.
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
0c 00000065`3804fac0 00000000`00000000 ntdll!RtlUserThreadStart+0x21
Unable to enumerate locals, Win32 error 0n87
Private symbols (symbols.pri) are required for locals.
Type ".hh dbgerr005" for details. --- 
Stay tuned. I keep working on complex MySQL/MariaDB problems under Windows, so soon will have few more findings and links to share.
Categories: Web Technologies

Why would I use a Webpack?

CSS-Tricks - Sat, 03/24/2018 - 16:45

Gonzalo García takes a crack at why webpack (not capitalized like npm) exists at all. No particular disagreements here, but here's my crack at it...

  • We use webpack because we need to import stuff from place;. This is a good pattern. We can use webpack to interpret those statements, as native support for them isn't what it needs to be yet, and it's not clear whether the native version will be smart for performance or not (probably not, at the scope of projects webpack is usually used for).
  • We use webpack because we know we need to concatenate and compress our JavaScript anyway, and managing load order isn't something you wanna handle manually.
  • We use webpack because of npm. Powerful features are a yarn or npm i away and so our projects are loaded with stuff to import.
  • We use webpack because we're sure it performs fancy magic that results in good performance-related things for our websites. We cross our fingers we have that right, and we've done our part right.
  • We use webpack because there is a hive mind in this industry and it leads to a lot of us hopping on the trains with the most people on them, and people are hanging out of the windows of the webpack train.

I'm very very (very) far from being a webpack expert, but I essentially get it, especially after the screencast Sean Larkin and I did right here, and I know enough my projects benefit from it.

Direct Link to ArticlePermalink

The post Why would I use a Webpack? appeared first on CSS-Tricks.

Categories: Web Technologies

Fun with Bugs #63 - On Bugs Detected by ASan

Planet MySQL - Sat, 03/24/2018 - 05:04
Among other things Geir Hoydalsvik stated in his nice post yesterday:
 "We’ve fixed a number of bugs detected by UBsan and Asan." This is indeed true, I already noted many related bugs fixed in recent MySQL 8.0.4. But I think that a couple of details are missing in the blog post. First of all, there still a notable number of bugs detected by ASan or noted in builds with ASan that remain "Verified". Second, who actually found and reported these bugs?

I decided to do a quick search and present my summary to clarify these details. Let me start with the list of "Verified" or "Open" bugs in public MySQL bugs database, starting from the oldest one:
  • Bug #69715 - "UBSAN: Item_func_mul::int_op() mishandles 9223372036854775809*-1". The oldest related "Verified" bug I found was reported back in 2013 by Arthur O'Dwyer. Shane Bester from Oracle kindly keeps checking it with recent and upcoming releases, so we know that even '9.0.0-dmr-ubsan' (built on 20 October 2017) was still affected.
  • Bug #80309 - "some innodb tests fail with address sanitizer (WITH_ASAN)". It was reported by Richard Prohaska and remains "Verified" for more than two years already.
  • Bug #80581 - "rpl_semi_sync_[non_]group_commit_deadlock crash on ASan, debug". This bug reported by Laurynas Biveinis from Percona two years ago is still "Verified".
  • Bug #81674 - "LeakSanitizer-enabled build fails to bootstrap server for MTR". This bug reported by  Laurynas Biveinis affects only MySQL 5.6, but still, why not to backport the fix from 5.7?
  • Bug #82026 - "Stack buffer overflow with --ssl-cipher=<more than 4K characters>". Bug detected by ASan was noted by Yura Sorokin from Percona and reported by Laurynas Biveinis.
  • Bug #82915 - "SIGKILL myself when using innodb_limit_optimistic_insert_debug=2 and drop table". ASan debug builds are affected. This bug was reported by Roel Van de Paar from Percona.
  • Bug #85995 - "Server error exit due to empty datadir causes LeakSanitizer errors". This bug in MySQL 8.0.1 (that had to affect anyone who runs tests on ASan debug builds on a regular basis) was reported by Laurynas Biveinis and stay "Verified" for almost a year.
  • Bug #87129 - "Unstable test main.basedir". This test problem reported by Laurynas Biveinis affects ASan builds, among others. See also his Bug #87190 - "Test main.group_by is unstable".
  • Bug #87201 - "XCode 8.3.3+ -DWITH_UBSAN=ON bundled protobuf build error". Yet another (this time macOS-specific) bug found by Laurynas Biveinis.
  • Bug #87295 - "Test group_replication.gr_single_primary_majority_loss_1 produces warnings". Potential bug in group replication noted by Laurynas Biveinis in ASan builds.
  • Bug #87923 - "ASan reporting a memory leak on merge_large_tests-t". This bug by Laurynas Biveinis is still "Verified", while Tor Didriksen's comment states that it it resolved with the fix for Bug #87922 (that is closed as fixed in MySQL 8.0.4). Why not to close this one also?
  • Bug #89438 - "LeakSanitizer errors on xplugin unit tests". As Laurynas Biveinis found, X Plugin unit tests report errors with LeakSanitizer.
  • Bug #89439 - "LeakSanitizer errors on GCS unit tests". yet another bug report for MySQL 8.0.4 by Laurynas Biveinis.
  • Bug #89961 - "add support for clang ubsan". This request was made by Tor Didriksen from Oracle. It is marked as "fixed in 8.0.12". It means we may get MySQL 8.0.11 released soon. That's why I decided to mention the bug here.
There were also few other test failures noted on ASan debug builds. I skipped them to make this post shorter.

Personally I do not run builds or tests with ASan on a regular basis. I appreciate Oracle's efforts to make code warning-free, UBSan- and ASan-clean, and fix bugs found with ASan. But I'd also want them to process all/most of related bugs in public database properly before making announcements of new related achievement, and clearly admit and appreciate a lot of help and contribution from specific community members (mostly Laurynas Biveinis in this case).

Percona engineers seem to test ASan builds of MySQL 5.7 and 8.0 (or Percona's closely related versions) regularly, for years, and contribute back public bug reports. I suspect they found way more related bugs than internal Oracle's QA. I think we should explicitly thank them for this contribution that made MySQL better!
Categories: Web Technologies

The spectrum of design roles in 2018

CSS-Tricks - Fri, 03/23/2018 - 15:45

Job titles is a regular topic around here. Occasionally heated, as job titles play a role in the hiring process (why are you asking me React questions for this UX design position role?). And complicated by the fact that there is no agreed-upon standards and the loads of people and companies who don't take them seriously (we just want people who do a good job, make your title whatever you want it to be). Complicated again when someone from the outside needs to look in.

Jasper Stephenson:

Recently, a colleague of mine named Mariko Sugita needed to hire a designer for a website she was creating. She’s an urbanist, and not particularly involved in the digital design field, so she asked the closest designer who happened to be on hand (me), “what kind of designers should I be looking for?”

Jasper takes a crack at it in this post.

Here's a roundup of other cracks at how to answer the same sort of question.

Direct Link to ArticlePermalink

The post The spectrum of design roles in 2018 appeared first on CSS-Tricks.

Categories: Web Technologies

End of an Era: Neither MC nor Continuent Attending Percona Live

Planet MySQL - Fri, 03/23/2018 - 08:23

Continuent have been a long term sponsor of the Percona Live conference, and the MySQL conference as it was before that, for many years. We have attended the conference both as a Diamond sponsor, and members of our staff attending and presenting our products and experience at the conference.

The nature of these conferences always changes over time, and we have seen over the last few years how the Percona Live conference has moved from being a pure MySQL conference to an open source database conference. Although Continuent continue to provide open source software and integrate with many open source databases, our core operation still revolves around MySQL clustering and replication for MySQL and Oracle.

Continuent is also evolving and changing and we are increasingly deploying and moving towards pure cloud-based environments, building and developing products that are used on the cloud or explicitly leverage cloud computing technology. We have a number of new products and initiatives specifically targeting these areas.

Over the course of the next year we will be releasing cloud editions of our clustering, replication and new backup and proxy services both directly and through our partners.

As such, this year we have made the difficult decision not to sponsor or attend the Percona Live conference, directing our energies to other conferences, webinars and meetups. We will be attending the AWS conference, for example, and we fully intend to be at some other select conferences this year dealing with analytics, in-memory computing, and cloud-based deployments.

To stay up-to-date with what Continuent are doing, keep reading the Continuent blog and follow us on Twitter and Facebook.

Categories: Web Technologies

Caching SHA-2 (or 256) Pluggable Authentication for MySQL 8

Planet MySQL - Fri, 03/23/2018 - 08:18
If you are like me and you spend chilly spring evenings relaxing by the fire, reading the manual for the upcoming MySQL 8 release, you may have seen Caching SHA-2 Pluggable Authentication in section 6.5.1.3. 

There are now TWO SHA-256 plugsins for MySQL 8 for hashing user account passwords and no, I do not know what the title of the manual pages says SHA-2 when it is SHA-256.  We have sha256_password for basic SHA-256 authentication and  caching_sha2_password that adds caching for better performance.

The default plugin is caching_sha2_password has three features not found in its non caching brother. The first is, predictably, a cache for faster authentication for repeat customers to the database. Next is a RSA-based password exchange that is independent of the SSL library you executable is linked. And it supports Unix socket-files and shared-memory protocols -- so sorry named pipe fans.

If you have been testing the release candidate and use older clients or older libmysqlclient you may have seen Authentication plugin 'caching_sha2_password' is not supported or some other similar message. You need updated clients to work with the updated server.  Old clients used the old MySQL native password password not the new chaching_ha2_password as the default.

When upgrading from 5,7,21 to the 8 GA version, existing accounts are not upgraded,  But if you are starting with a fresh install you get the chaching_sha2_password in your mysql.user entry.   I am sure this will catch someone so please take care. And this goes for new replication servers too! 
Categories: Web Technologies

Figma Web API

CSS-Tricks - Fri, 03/23/2018 - 06:50

Figma launched their Web Platform API which allows developers to build on top of and extend the features of their web-based design tool. They also published a quick post about the release that showcases how design teams at Uber and GitHub are starting to use the API but they also also dig into a few public extensions that are available to use today.

Direct Link to ArticlePermalink

The post Figma Web API appeared first on CSS-Tricks.

Categories: Web Technologies

Approaches to Deprecating Code in JavaScript

CSS-Tricks - Fri, 03/23/2018 - 06:43

Recently, I had to dig into the topic of code deprecation in JavaScript. I feel like this topic gets less coverage even though it may a play key role in certain projects, especially when working in bigger teams or dealing with external APIs.

In JavaScript-land, I don't know of any true industry standards for deprecating JavaScript. It could be different per any team, library or vendor.

That’s why my goal here is to sum up my findings and thoughts on this topic, alongside some good practices when it’s time to mark a JavaScript method obsolete a codebase.

What does “deprecation” actually mean?

First, let’s start by clarifying that the deprecation is just a status applied to a software feature. It indicates that this feature should be avoided, typically because it has been superseded.

Deprecation may also indicate that the feature will be removed in the future. Features are deprecated—rather than immediately removed—in order to provide backward compatibility, and give programmers who have used the feature time to bring their code into compliance with the new standard.

Additionally, a deprecated feature suggests that there won’t be any further development from this point onward. It shouldn’t work any different than it did in a previous version (unless documentation explicitly states something else). So, generally, it should be the same as it was when the deprecation action happened.

It may or may not work in the latest version—no guarantees!

However, since there aren’t any true industry standards that are strictly followed inJavaScript-land, this could be slightly different per team, library or vendor.

When to deprecate code and when to delete it?

It’s important to note that a deprecated software feature or method is still a part of the software! Consider the "deprecated" label as just a status of the code. Whether the software feature will actually be removed in the future depends on what that particular software team decides.

In my opinion, large teams or projects relying on external APIs or libraries ought to deprecate first, then remove later (after a reasonable time, however you define that). At the very least, give at least one major version bump before actually removing the deprecated code so users have a chance to adjust to the change.

You might want to look at Semantic Versioning, a simple set of rules and requirements that dictate how version numbers are assigned and incremented. Given a version number MAJOR.MINOR.PATCH, increment the MAJOR version when you make incompatible API changes, MINOR version when you add functionality in a backwards-compatible manner, and PATCH version when you make backwards-compatible bug fixes.

If your software is rapidly changing and evolving and you are deprecating a feature, try to communicate with your project manager if this feature is expected to be resurrected later. If you choose to deprecate, instead of delete, it might be a lot easier for you to revert should you need to.

For smaller teams or projects with internal methods and APIs, go ahead and remove first rather than deprecate. Sometimes it just doesn’t make sense to waste time and deprecation only increases the complexity just for the sake of following best practices.

How to mark a method obsolete

Here are five good practices I have found the most useful:

  1. Add a @deprecated JSDoc flag.
  2. Mention the version that the method was deprecated.
  3. Figure out a timeframe for when this method will be deleted, including which version it will take place. Otherwise, based on my experience, it stays forever &#x1f642;
  4. Use comments liberally to explain the implementation for the benefit of other developers or your future self. This is extremely useful if your use-case is writing a library that others use as a dependency for their work.
  5. Add a console warning message that indicates that the function is deprecated.

Here’s a more practical example where I use all five practices:

/** * A magic method that multiples digits. * * @deprecated [#1] since version 2.3 [#2]. * [#3] Will be deleted in version 3.0. * [#4] In case you need similar behavior, implement it on you own, * preferably in vanilla JavaScript * or use the multiplyTheSameNumber method instead, * if the same number needs to be multiplied multiple times, like so: * multiplyDigits([5, 5, 5]) === multiplyTheSameNumber(5, 3) * * @param {array} _digits - digits to multiply */ function multiplyDigits(_digits) { console.warn("Calling a depricated method!"); // [#5] // .... }

To avoid repetition in the console warnings or in case you plan to deprecate multiple methods and you have their replacements, it might be more convenient to use a helper:

/** * Creating a deprecated / obsolete behavior for methods in a library. * [Credits]{@link: https://stackoverflow.com/q/21726472/1333836} * * @param {function} replacementFunction * @param {string} oldFnName * @param {string} newFnName * @return {function} */ const Oboslete = function(replacementFunction, oldFnName, newFnName) { const wrapper = function() { console.warn("WARNING! Obsolete function called. Function '" + oldFnName + "' has been deprecated, please use the new '" + newFnName + "' function instead!"); replacementFunction.apply(this, arguments); } wrapper.prototype = replacementFunction.prototype; return wrapper; } Wrapping up

I’d suggest getting your team on the same page and inherit deprecation practices that make the most sense for your project or use case, whether it’s adopting the practices we’ve covered here or others.

Note that there are certain times when deletion makes more sense than deprecation. Sometimes, investing efforts to deprecate something simply aren’t worth it. Again, it’s totally up to you and what makes the most sense for your project.

Do you know other good practices when marking a method obsolete in JavaScript? Let me know in the comments!

Credits

The ideas I shared here were inspired by comments I found on Software Engineering Stack Exchange and on StackOverflow.

The post Approaches to Deprecating Code in JavaScript appeared first on CSS-Tricks.

Categories: Web Technologies

MySQL 8.0 Source Code Improvements

Planet MySQL - Fri, 03/23/2018 - 05:52

With this post, I want to bring your attention to source code improvements in MySQL 8.0. MySQL 8.0 modernizes the code base by using C++11 constructs, being warning-free on more compilers and platforms, being UBSan- and ASan- clean, improving header file dependencies, improving the coding style, and better developer documentation.…

Categories: Web Technologies

MySQL 8.0 Source Code Improvements

MySQL Server Blog - Fri, 03/23/2018 - 05:52

With this post, I want to bring your attention to source code improvements in MySQL 8.0. MySQL 8.0 modernizes the code base by using C++11 constructs, being warning-free on more compilers and platforms, being UBSan- and ASan- clean, improving header file dependencies, improving the coding style, and better developer documentation.…

Categories: Web Technologies

This Week in Data with Colin Charles 33: Reporting from FOSSASIA 2018 and Azure Announces Database Services for MySQL and PostgreSQL

Planet MySQL - Fri, 03/23/2018 - 02:16

Join Percona Chief Evangelist Colin Charles as he covers happenings, gives pointers and provides musings on the open source database community.

Writing to you on the ground from FOSSASIA 2018, where I gave a track introduction yesterday since we have a pretty awesome database track most Saturday, and generally, all MySQL focused on Sunday. There’s even a list of talks by Oracle MySQL’ers (yes, there’s more than just Oracle folk, but for that, you got to get the schedule).

The Percona Live Community Dinner happens again this year during Percona Live 2018, at Pedro’s on 24 April 2018. It starts at 7 pm, and I highly recommend you purchase the $30 ticket. It usually sells out, so don’t wait till it’s too late.

Some big news from a MySQL in the cloud perspective: Announcing general availability of Azure database services for MySQL and PostgreSQL. There’s also a quick guide: Create an Azure Database for MySQL server by using the Azure portal. Note that next comes MariaDB Server too.

Releases Link List Upcoming appearances

The post This Week in Data with Colin Charles 33: Reporting from FOSSASIA 2018 and Azure Announces Database Services for MySQL and PostgreSQL appeared first on Percona Database Performance Blog.

Categories: Web Technologies

Solving Problems Using Trees - Nomad PHP

Planet PHP - Thu, 03/22/2018 - 21:05

June 2018 - EU
Presented By
Tomasz Kowalczyk
June 21, 2018
20:00 CEST

The post Solving Problems Using Trees appeared first on Nomad PHP.

Categories: Web Technologies

Shinguz: MySQL sys Schema in MariaDB 10.2

Planet MySQL - Thu, 03/22/2018 - 14:54

MySQL has introduced the PERFORMANCE_SCHEMA (P_S) in MySQL 5.5 and made it really usable in MySQL 5.6 and added some enhancements in MySQL 5.7 and 8.0.

Unfortunately the PERFORMANCE_SCHEMA was not really intuitive for the broader audience. Thus Mark Leith created the sys Schema for an easier access for the normal DBA and DevOps and Daniel Fischer has enhanced it further. Fortunately the sys Schema up to version 1.5.1 is available on GitHub. So we can adapt and use it for MariaDB as well. The version of the sys Schema in MySQL 8.0 is 1.6.0 and seems not to be on GitHub yet. But you can extract it from the MySQL 8.0 directory structure: mysql-8.0/share/mysql_sys_schema.sql. According to a well informed source the project on GitHub is not dead but the developers have just been working on other priorities. An the source announced another release soon (they are working on it at the moment).

MariaDB has integrated the PERFORMANCE_SCHEMA based on MySQL 5.6 into its own MariaDB 10.2 server but unfortunately did not integrate the sys Schema. Which PERFORMANCE_SCHEMA version is integrated in MariaDB can be found here.

To install the sys Schema into MariaDB we first have to check if the PERFORMANCE_SCHEMA is activated in the MariaDB server:

mariadb> SHOW GLOBAL VARIABLES LIKE 'performance_schema'; +--------------------+-------+ | Variable_name | Value | +--------------------+-------+ | performance_schema | OFF | +--------------------+-------+

To enable the PERFORMANCE_SCHEMA just add the following line to your my.cnf:

[mysqld] performance_schema = 1

and restart the instance.

In MariaDB 10.2 the MySQL 5.6 PERFORMANCE_SCHEMA is integrated so we have to run the sys_56.sql installation script. If you try to run the sys_57.sql script you will get a lot of errors...

But also the sys_56.sql installation script will cause you some little troubles which are easy to fix:

unzip mysql-sys-1.5.1.zip mysql -uroot < sys_56.sql ERROR 1193 (HY000) at line 20 in file: './procedures/diagnostics.sql': Unknown system variable 'server_uuid' ERROR 1193 (HY000) at line 20 in file: './procedures/diagnostics.sql': Unknown system variable 'master_info_repository' ERROR 1193 (HY000) at line 20 in file: './procedures/diagnostics.sql': Unknown system variable 'relay_log_info_repository'

For a quick hack to make the sys Schema work I changed the following information:

  • server_uuid to server_id
  • @@master_info_repository to NULL (3 times).
  • @@relay_log_info_repository to NULL (3 times).

For the future the community has to think about if the sys Schema should be aware of the 2 branches MariaDB and MySQL and act accordingly or if the sys Schema has to be forked to work properly for MariaDB and implement MariaDB specific functionality.

When the sys Schema finally is installed you have the following tables to get your performance metrics:

mariadb> use sys mariadb> SHOW TABLES; +-----------------------------------------------+ | Tables_in_sys | +-----------------------------------------------+ | host_summary | | host_summary_by_file_io | | host_summary_by_file_io_type | | host_summary_by_stages | | host_summary_by_statement_latency | | host_summary_by_statement_type | | innodb_buffer_stats_by_schema | | innodb_buffer_stats_by_table | | innodb_lock_waits | | io_by_thread_by_latency | | io_global_by_file_by_bytes | | io_global_by_file_by_latency | | io_global_by_wait_by_bytes | | io_global_by_wait_by_latency | | latest_file_io | | metrics | | processlist | | ps_check_lost_instrumentation | | schema_auto_increment_columns | | schema_index_statistics | | schema_object_overview | | schema_redundant_indexes | | schema_table_statistics | | schema_table_statistics_with_buffer | | schema_tables_with_full_table_scans | | schema_unused_indexes | | session | | statement_analysis | | statements_with_errors_or_warnings | | statements_with_full_table_scans | | statements_with_runtimes_in_95th_percentile | | statements_with_sorting | | statements_with_temp_tables | | sys_config | | user_summary | | user_summary_by_file_io | | user_summary_by_file_io_type | | user_summary_by_stages | | user_summary_by_statement_latency | | user_summary_by_statement_type | | version | | wait_classes_global_by_avg_latency | | wait_classes_global_by_latency | | waits_by_host_by_latency | | waits_by_user_by_latency | | waits_global_by_latency | +-----------------------------------------------+

One query as an example: Top 10 MariaDB global I/O latency files on my system:

mariadb> SELECT * FROM sys.waits_global_by_latency LIMIT 10; +--------------------------------------+-------+---------------+-------------+-------------+ | events | total | total_latency | avg_latency | max_latency | +--------------------------------------+-------+---------------+-------------+-------------+ | wait/io/file/innodb/innodb_log_file | 112 | 674.18 ms | 6.02 ms | 23.75 ms | | wait/io/file/innodb/innodb_data_file | 892 | 394.60 ms | 442.38 us | 29.74 ms | | wait/io/file/sql/FRM | 668 | 72.85 ms | 109.05 us | 20.17 ms | | wait/io/file/sql/binlog_index | 10 | 21.25 ms | 2.13 ms | 15.74 ms | | wait/io/file/sql/binlog | 19 | 11.18 ms | 588.56 us | 10.38 ms | | wait/io/file/myisam/dfile | 79 | 10.48 ms | 132.66 us | 3.78 ms | | wait/io/file/myisam/kfile | 86 | 7.23 ms | 84.01 us | 789.44 us | | wait/io/file/sql/dbopt | 35 | 1.95 ms | 55.61 us | 821.68 us | | wait/io/file/aria/MAI | 269 | 1.18 ms | 4.40 us | 91.20 us | | wait/io/table/sql/handler | 36 | 710.89 us | 19.75 us | 125.37 us | +--------------------------------------+-------+---------------+-------------+-------------+
Taxonomy upgrade extras:  mariadb sys performance_schema 10.2
Categories: Web Technologies

MySQL Log Rotation

Planet MySQL - Thu, 03/22/2018 - 08:00
Overview

I find far too often that MySQL error and slow query logs are unaccounted for.  Setting up log rotation helps make the logs manageable in the event that they start to fill up and can help make your troubleshooting of issues more efficient.

Setup

All steps in the examples below are run as the root user. The first step is to setup a user that will perform the log rotation.  It is recommended to only give enough access to the MySQL user for the task that it is performing.

Create Log Rotate MySQL User

mysql > CREATE USER 'log_rotate'@'localhost' IDENTIFIED BY '<ENTER PASSWORD HERE>'; mysql > GRANT RELOAD,SUPER ON *.* to 'log_rotate'@'localhost'; mysql > FLUSH PRIVILEGES;</pre>

 

The next step is to setup the MySQL authentication config as root.  Here are two methods to set this up.  The first method will be the more secure method of storing your MySQL credentials using mysql_config_editor as the credentials are stored encrypted. But this first method is only available with Oracle MySQL or Percona MySQL client greater than 5.5. It is not available with Maria DB MySQL client. Method 2 can be used with pretty much any setup but is less secure as the password is stored in plain text.

 

Method 1

bash # mysql_config_editor set \ --login-path=logrotate \ --host=localhost \ --user=log_rotate \ --port 3306 \ --password

Method 2

bash # vi /root/.my.cnf [client] user=log_rotate password='<ENTER PASSWORD HERE>' bash # chmod 600 /root/.my.cnf

 

Now we will test to make sure this is working as expected

 

Method 1

bash # mysqladmin --login-path=logrotate ping

Method 2

bash # mysqladmin ping

 

The paths for the error and slow query log will need to be gathered in order to place them in the logrotate config file

 

Method 1

bash # mysql --login-path=logrotate -e "show global variables like 'slow_query_log_file'" bash # mysql --login-path=logrotate -e "show global variables like 'log_error'"

Method 2

bash # mysql -e "show global variables like 'slow_query_log_file'" bash # mysql -e "show global variables like 'log_error'"

 

Finally we will create the log rotation file with the following content. Make sure to update your error and slow query log paths to match the paths gathered in previous steps. Start by opening up the editor for a new mysql logrotate file in the /etc/logrotate.d directory.

 

Method 1 Content

bash # vi /etc/logrotate.d/mysql /var/lib/mysql/error.log /var/lib/mysql/slow.queries.log { create 600 mysql mysql daily rotate 30 missingok compress sharedscripts postrotate if test -x /usr/bin/mysqladmin && env HOME=/root /usr/bin/mysqladmin --login-path=logrotate ping > /dev/null 2>&1 then env HOME=/root/ /usr/bin/mysql --login-path=logrotate -e 'select @@global.long_query_time into @lqt_save; set global long_query_time=2000; set global slow_query_log=0; select sleep(2); FLUSH ERROR LOGS; FLUSH SLOW LOGS;select sleep(2); set global long_query_time=@lqt_save; set global slow_query_log=1;' > /var/log/mysqladmin.flush-logs 2>&1 fi endscript }

Method 2 Content

bash # vi /etc/logrotate.d/mysql /var/lib/mysql/error.log /var/lib/mysql/slow.queries.log { create 600 mysql mysql daily rotate 30 missingok compress sharedscripts postrotate if test -x /usr/bin/mysqladmin && env HOME=/root /usr/bin/mysqladmin ping > /dev/null 2>&1 then env HOME=/root/ /usr/bin/mysql -e 'select @@global.long_query_time into @lqt_save; set global long_query_time=2000; set global slow_query_log=0; select sleep(2); FLUSH ERROR LOGS; FLUSH SLOW LOGS;select sleep(2); set global long_query_time=@lqt_save; set global slow_query_log=1;' > /var/log/mysqladmin.flush-logs 2>&1 fi endscript } Validation

For final validation force a log rotate. Update the path in the ls command to match the path of the logs gathered earlier.

bash # logrotate --force /etc/logrotate.d/mysql bash # ls -al /var/lib/mysql
Categories: Web Technologies

Good ol’ Margin Collapsing

CSS-Tricks - Thu, 03/22/2018 - 06:48

Here's a one-sentence blog post about margin collapsing: When two block elements are stacked on top of one another, the vertical space between them is the larger between the one on top's margin-bottom and the one on the bottom's margin-top.

It's a bit weird and ascii-shruggy. Couple caveats with them, as you might expect.

I find it fascinating how it makes for the perfect sort of mini blog post when people have an ah-ha moment about it. MDN even sees fit to have a dedicated page.

Adam Roberts in 2015:

Although the margin collapse behavior is at first a little unintuitive, it does make life easier in the case of multiple nested elements, where the behavior is often desirable.

Andrew Grant in 2015:

It's probably not the most intuitive aspect of CSS, but the takeaway here is that there is some logic at play and, once you have learned it, the mystery and confusion suddenly disappears!

Geoff Graham in 2015:

Do you see how collapsing margins can make things tricky? I personally encounter this on a frustratingly frequent basis when dealing with typography.

Magnus Benoni in 2016:

Margin collapsing can be frustrating to deal with, but knowing when and how it happens will make it easier for you to fix problems when they occur.

Ire Aderinokun in 2017:

Collapsible margins can be a pain if you don't properly understand when they occur. The first step to dealing with or avoiding them is [to] understand exactly which case of collapsible margins we are dealing with.

Adam Laki in 2018:

Margin collapse is something that mentioned in every CSS book’s first or second chapter. When I learned about the stylesheets in a long time ago, of course, I read about it.

Jonathan Harrell in 2018:

The concept of margin collapse is foundational to an understanding of the box model in CSS, but it is actually quite complex and potentially confusing. The spec describing how margin collapse works is thorough but difficult to understand.

I don't point all these out to say it's an over-blogged subject (nothing is), but that it's an interesting thread to follow. When so many people feel the need to explain something ultimately so small, there is something weird (probably bad) happening there.

No doubt this is why it's considered one of the mistakes in the design of CSS:

The top and bottom margins of a box should never have been allowed to collapse together automatically as this is the root of all margin-collapsing evil.

Emphasis theirs and is, in fact, the only bold text in the entire list.

If you're looking to stop the behavior, you'll need to do something that probably has more side effects than it's worth, like float the elements. It's not quite as simple as kicking off a new Block Formatting Context, as that's what display: flow-root; does and that doesn't work.

You're probably better off just knowing about it and dealing with it as it comes up by systematically flowing margins in one direction, or getting weird.

The post Good ol’ Margin Collapsing appeared first on CSS-Tricks.

Categories: Web Technologies

​Deliver exceptional customer experiences in your product

CSS-Tricks - Thu, 03/22/2018 - 06:48

(This is a sponsored post.)

​Pendo delivers the only complete platform for product teams that helps companies create great products. The Pendo Product Experience Platform enables product teams to understand product usage, collect user feedback, measure NPS, assist users in their apps and promote new features in product - all without requiring any engineering resources. This unique combination of capabilities is all built on a common infrastructure of product data and results in improved customer satisfaction, reduced churn, and increased revenue.

Pendo is the proven choice of innovative product leaders at Salesforce, Proofpoint, Optimizely, Citrix, BMC and many more leading companies.

Request a demo of Pendo today.​

Direct Link to ArticlePermalink

The post ​Deliver exceptional customer experiences in your product appeared first on CSS-Tricks.

Categories: Web Technologies

Docker Compose Setup for InnoDB Cluster

Planet MySQL - Thu, 03/22/2018 - 05:07
In the following we show how InnoDB cluster can be deployed in a container context. In the official documentation (Introducing InnoDB Cluster), InnoDB is described as: MySQL InnoDB cluster provides a complete high availability solution for MySQL. MySQL Shell includes AdminAPI which enables you to easily configure and administer a group of at least three […]
Categories: Web Technologies

How to install XMB forum on Ubuntu 16.04 LTS

Planet MySQL - Thu, 03/22/2018 - 04:05
XMB forum also known as eXtreme Message Board is a free and open source forum software written in PHP and uses MySQL database backend. XMB is a simple, lightweight, easy to use, Powerful and highly customizable. In this tutorial, we will learn how to install XMB forum on Ubuntu 16.04.
Categories: Web Technologies

The programming languages you should learn now

InfoWorld JavaScript - Thu, 03/22/2018 - 03:00

Learning a programming language is not hard. In fact, if you’re experienced, you can learn the basics in under 24 hours. So if you’re in the market for a new lingua franca, such as to bolster your hirability, what you choose next might be influenced by your current language of choice.

Here are the languages I suggest you consider learning if you don’t already know them, based on the languages you already know.

[ Learn how your enterprise can excel in agile development. | Take your agile career to the next level: How to improve your scrum master skills. | Not sure what “agile” really means? InfoWorld explains the agile methodoology. | Keep up with hot topics in programming with InfoWorld’s App Dev Report newsletter. ] If you know Java, choose from these languages InfoWorld has you covered for all the hot languages

Go (Golang):

To read this article in full, please click here

Categories: Web Technologies

FLUSH and LOCK Handling in Percona XtraDB Cluster

Planet MySQL - Wed, 03/21/2018 - 12:30

In this blog post, we’ll look at how Percona XtraDB Cluster (PXC) executes FLUSH and LOCK handling.

Introduction

Percona XtraDB Cluster is a multi-master solution that allows parallel execution of the transactions on multiple nodes at the same point in time. Given this semantics, it is important to understand how Percona XtraDB Cluster executes statements regarding FLUSH and LOCK handling (that operate at node level).

The section below enlist different flavors of these statements and their PXC semantics

FLUSH TABLE WITH READ LOCK
  • FTWRL is normally used for backup purposes.
  • Execution of this command establishes a global level read lock.
  • This read lock is non-preemptable by the background running applier thread.
  • PXC causes the node to move to DESYNC state (thereby blocking emission of flow-control) and also pauses the node.

2018-03-08T05:09:54.293991Z 0 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 1777) 2018-03-08T05:09:58.040809Z 5 [Note] WSREP: Provider paused at c7daf065-2285-11e8-a848-af3e3329ab8f:2002 (2047) 2018-03-08T05:14:20.508317Z 5 [Note] WSREP: resuming provider at 2047 2018-03-08T05:14:20.508350Z 5 [Note] WSREP: Provider resumed. 2018-03-08T05:14:20.508887Z 0 [Note] WSREP: Member 1.0 (n2) resyncs itself to group 2018-03-08T05:14:20.508900Z 0 [Note] WSREP: Shifting DONOR/DESYNCED -> JOINED (TO: 29145) 2018-03-08T05:15:16.932759Z 0 [Note] WSREP: Member 1.0 (n2) synced with group. 2018-03-08T05:15:16.932782Z 0 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 29145) 2018-03-08T05:15:16.988029Z 2 [Note] WSREP: Synchronized with group, ready for connections 2018-03-08T05:15:16.988054Z 2 [Note] WSREP: Setting wsrep_ready to true

  • Other nodes of the cluster continue to process the workload.
  • DESYNC and pause node continue to see the replication traffic. Though it doesn’t process the write-sets, they are appended to Galera cache for future processing.
  • Fallback: When FTWRL is released (through UNLOCK TABLES), and if the workload is active on other nodes of the cluster, FTWRL executed node may start emitting flow-control to cover the backlog. Check details here.
FLUSH TABLE <tablename> (WITH READ LOCK|FOR EXPORT)
  • It is meant to take global level read lock on the said table only. This lock command is not replicated and so pxc_strict_mode = ENFORCING blocks execution of this command.
  • This read lock is non-preemptable by the background running applier thread.
  • Execution of this command will cause the node to pause.
  • If the flush command executing node is same as workload processing node, then the node will pause immediately
  • If the flush command executing node is different from workload processing node, then the write-sets are queued to the incoming queue and flow-control will cause the pause.
  • End-result is cluster will stall in both cases.

2018-03-07T06:40:00.143783Z 5 [Note] WSREP: Provider paused at 40de14ba-21be-11e8-8e3d-0ee226700bda:147682 (149032) 2018-03-07T06:40:00.144347Z 5 [Note] InnoDB: Sync to disk of `test`.`t` started. 2018-03-07T06:40:00.144365Z 5 [Note] InnoDB: Stopping purge 2018-03-07T06:40:00.144468Z 5 [Note] InnoDB: Writing table metadata to './test/t.cfg' 2018-03-07T06:40:00.144537Z 5 [Note] InnoDB: Table `test`.`t` flushed to disk 2018-03-07T06:40:01.855847Z 5 [Note] InnoDB: Deleting the meta-data file './test/t.cfg' 2018-03-07T06:40:01.855874Z 5 [Note] InnoDB: Resuming purge 2018-03-07T06:40:01.855955Z 5 [Note] WSREP: resuming provider at 149032 2018-03-07T06:40:01.855970Z 5 [Note] WSREP: Provider resumed.

  • Once the lock is released (through UNLOCK TABLES), node resumes apply of write-sets.
LOCK TABLE <tablename> READ/WRITE
  • LOCK TABLE command is meant to lock the said table in the said mode.
  • Again, the lock established by this command is non-preemptable.
  • LOCK is taken at node level (command is not replicated) so pxc_strict_mode = ENFORCING blocks this command.
  • There is no state change in PXC on the execution of this command.
  • If the lock is taken on the table that is not being touched by the active workload, the workload can continue to progress. If the lock is taken on the table that is part of the workload, said transaction in the workload will wait for the lock to get released, in turn, will cause complete workload to halt.
GET_LOCK
  • It is named lock and follows same semantics as LOCK TABLE for PXC. (Base semantics of MySQL are slightly different that you can check here).
LOCK TABLES FOR BACKUP
  • As the semantics goes, this lock is specially meant for backup and blocks non-transactional changes (like the updates to non-transactional engine = MyISAM and DDL changes).
  • PXC doesn’t have any special add-on semantics for this command
LOCK BINLOG FOR BACKUP
  • This statement blocks write to binlog. PXC always generates a binlog (persist to disk is controlled by the log-bin setting). If you disable log-bin, then PXC enables emulation-based binlogging.
  • This effectively means this command can cause the cluster to stall.
Tracking active lock/flush
  • If you have executed a flush or lock command and wanted to find out, it is possible using the com_% counter. These counters are connection specific, so execute these commands from the same client connection. Also, these counters are aggregate counters and incremental only.

mysql> show status like 'Com%lock%'; +----------------------------+-------+ | Variable_name | Value | +----------------------------+-------+ | Com_lock_tables | 2 | | Com_lock_tables_for_backup | 1 | | Com_lock_binlog_for_backup | 1 | | Com_unlock_binlog | 1 | | Com_unlock_tables | 5 | +----------------------------+-------+ 5 rows in set (0.01 sec) mysql> show status like '%flush%'; +--------------------------------------+---------+ | Variable_name | Value | +--------------------------------------+---------+ | Com_flush | 4 | | Flush_commands | 3 | * Flush_commands is a global counter. Check MySQL documentation for more details.

Conclusion

By now, we can conclude that the user should be a bit more careful when executing local lock commands (understanding the semantics and the effect). Careful execution of these commands can help serve your purpose.

The post FLUSH and LOCK Handling in Percona XtraDB Cluster appeared first on Percona Database Performance Blog.

Categories: Web Technologies

Pages