ndbd is the process that is used to handle all the data in tables using the NDB Cluster storage engine. This is the process that empowers a data node to accomplish distributed transaction handling, node recovery, checkpointing to disk, online backup, and related tasks.

In a MySQL Cluster, a set of ndbd processes cooperate in handling data. These processes can execute on the same computer (host) or on different computers. The correspondences between data nodes and Cluster hosts is completely configurable.

The following table includes command options specific to the MySQL Cluster data node program ndbd. Additional descriptions follow the table. For options common to most MySQL Cluster programs (including ndbd), see Section 17.4.24, "Options Common to MySQL Cluster Programs - Options Common to MySQL Cluster Programs".

ndbd generates a set of log files which are placed in the directory specified by DataDir in the config.ini configuration file.

These log files are listed below. node_id is the node's unique identifier. Note that node_id represents the node's unique identifier. For example, ndb_2_error.log is the error log generated by the data node whose node ID is 2.

It is recommended not to use a directory mounted through NFS because in some environments this can cause problems whereby the lock on the .pid file remains in effect even after the process has terminated.

To start ndbd, it may also be necessary to specify the host name of the management server and the port on which it is listening. Optionally, one may also specify the node ID that the process is to use.

shell> ndbd --connect-string="nodeid=2;host=ndb_mgmd.mysql.com:1186"

See Section 17.3.2.3, "The MySQL Cluster Connectstring", for additional information about this issue. Section 17.4.1, "ndbd - The MySQL Cluster Data Node Daemon", describes other options for ndbd.

When ndbd starts, it actually initiates two processes. The first of these is called the "angel process"; its only job is to discover when the execution process has been completed, and then to restart the ndbd process if it is configured to do so. Thus, if you attempt to kill ndbd using the Unix kill command, it is necessary to kill both processes, beginning with the angel process. The preferred method of terminating an ndbd process is to use the management client and stop the process from there.

The execution process uses one thread for reading, writing, and scanning data, as well as all other activities. This thread is implemented asynchronously so that it can easily handle thousands of concurrent actions. In addition, a watch-dog thread supervises the execution thread to make sure that it does not hang in an endless loop. A pool of threads handles file I/O, with each thread able to handle one open file. Threads can also be used for transporter connections by the transporters in the ndbd process. In a multi-processor system performing a large number of operations (including updates), the ndbd process can consume up to 2 CPUs if permitted to do so.

For a machine with many CPUs it is possible to use several ndbd processes which belong to different node groups; however, such a configuration is still considered experimental and is not supported for MySQL 5.5 in a production setting. See Section 17.1.6, "Known Limitations of MySQL Cluster".

ndbinfo_select_all is a client program that selects all rows and columns from one or more tables in the ndbinfo database.

Not all ndbinfo tables can be accessed by this program. ndbinfo_select_all can access the counters, diskpagebuffer, logbuffers, logspaces, nodes, resources, threadblocks, threadstat, and transporters tables.

To select from one or more ndbinfo tables using ndbinfo_select_all, it is necessary to supply the names of the tables when invoking the program as shown here:

shell> ndbinfo_select_all table_name1  [table_name2] [...]

For example:

shell> ndbinfo_select_all logbuffers logspaces== logbuffers ==node_id log_type log_id  log_part total   used high5   0   0   0   33554432 262144  06   0   0   0   33554432 262144  07   0   0   0   33554432 262144  08   0   0   0   33554432 262144  0== logspaces ==node_id log_type log_id  log_part total   used high5   0   0   0   268435456   0   05   0   0   1   268435456   0   05   0   0   2   268435456   0   05   0   0   3   268435456   0   06   0   0   0   268435456   0   06   0   0   1   268435456   0   06   0   0   2   268435456   0   06   0   0   3   268435456   0   07   0   0   0   268435456   0   07   0   0   1   268435456   0   07   0   0   2   268435456   0   07   0   0   3   268435456   0   08   0   0   0   268435456   0   08   0   0   1   268435456   0   08   0   0   2   268435456   0   08   0   0   3   268435456   0   0shell>   

The following table includes options that are specific to ndbinfo_select_all. Additional descriptions follow the table. For options common to most MySQL Cluster programs (including ndbinfo_select_all), see Section 17.4.24, "Options Common to MySQL Cluster Programs - Options Common to MySQL Cluster Programs".

ndbmtd is a multi-threaded version of ndbd, the process that is used to handle all the data in tables using the NDBCLUSTER storage engine. ndbmtd is intended for use on host computers having multiple CPU cores. Except where otherwise noted, ndbmtd functions in the same way as ndbd; therefore, in this section, we concentrate on the ways in which ndbmtd differs from ndbd, and you should consult Section 17.4.1, "ndbd - The MySQL Cluster Data Node Daemon", for additional information about running MySQL Cluster data nodes that apply to both the single-threaded and multi-threaded versions of the data node process.

Command-line options and configuration parameters used with ndbd also apply to ndbmtd. For more information about these options and parameters, see Section 17.4.1, "ndbd - The MySQL Cluster Data Node Daemon", and Section 17.3.2.6, "Defining MySQL Cluster Data Nodes", respectively.

ndbmtd is also file system-compatible with ndbd. In other words, a data node running ndbd can be stopped, the binary replaced with ndbmtd, and then restarted without any loss of data. (However, when doing this, you must make sure that MaxNoOfExecutionThreads is set to an apppriate value before restarting the node if you wish for ndbmtd to run in multi-threaded fashion.) Similarly, an ndbmtd binary can be replaced with ndbd simply by stopping the node and then starting ndbd in place of the multi-threaded binary. It is not necessary when switching between the two to start the data node binary using --initial.

Using ndbmtd differs from using ndbd in two key respects:

Like ndbd, ndbmtd generates a set of log files which are placed in the directory specified by DataDir in the config.ini configuration file. Except for trace files, these are generated in the same way and have the same names as those generated by ndbd.

In the event of a critical error, ndbmtd generates trace files describing what happened just prior to the error' occurrence. These files, which can be found in the data node's DataDir, are useful for analysis of problems by the MySQL Cluster Development and Support teams. One trace file is generated for each ndbmtd thread. The names of these files have the following pattern:

  ndb_node_id_trace.log.trace_id_tthread_id,

In this pattern, node_id stands for the data node's unique node ID in the cluster, trace_id is a trace sequence number, and thread_id is the thread ID. For example, in the event of the failure of an ndbmtd process running as a MySQL Cluster data node having the node ID 3 and with MaxNoOfExecutionThreads equal to 4, four trace files are generated in the data node's data directory. If the is the first time this node has failed, then these files are named ndb_3_trace.log.1_t1, ndb_3_trace.log.1_t2, ndb_3_trace.log.1_t3, and ndb_3_trace.log.1_t4. Internally, these trace files follow the same format as ndbd trace files.

The ndbd exit codes and messages that are generated when a data node process shuts down prematurely are also used by ndbmtd. See ndbd Error Messages, for a listing of these.

The management server is the process that reads the cluster configuration file and distributes this information to all nodes in the cluster that request it. It also maintains a log of cluster activities. Management clients can connect to the management server and check the cluster's status.

The following table includes options that are specific to the MySQL Cluster management server program ndb_mgmd. Additional descriptions follow the table. For options common to most MySQL Cluster programs (including ndb_mgmd), see Section 17.4.24, "Options Common to MySQL Cluster Programs - Options Common to MySQL Cluster Programs".

It is not strictly necessary to specify a connectstring when starting the management server. However, if you are using more than one management server, a connectstring should be provided and each node in the cluster should specify its node ID explicitly.

See Section 17.3.2.3, "The MySQL Cluster Connectstring", for information about using connectstrings. Section 17.4.4, "ndb_mgmd - The MySQL Cluster Management Server Daemon", describes other options for ndb_mgmd.

The following files are created or used by ndb_mgmd in its starting directory, and are placed in the DataDir as specified in the config.ini configuration file. In the list that follows, node_id is the unique node identifier.

The ndb_mgm management client process is actually not needed to run the cluster. Its value lies in providing a set of commands for checking the cluster's status, starting backups, and performing other administrative functions. The management client accesses the management server using a C API. Advanced users can also employ this API for programming dedicated management processes to perform tasks similar to those performed by ndb_mgm.

To start the management client, it is necessary to supply the host name and port number of the management server:

shell> ndb_mgm [host_name [port_num]]

For example:

shell> ndb_mgm ndb_mgmd.mysql.com 1186

The default host name and port number are localhost and 1186, respectively.

The following table includes options that are specific to the MySQL Cluster management client program ndb_mgm. Additional descriptions follow the table. For options common to most MySQL Cluster programs (including ndb_mgm), see Section 17.4.24, "Options Common to MySQL Cluster Programs - Options Common to MySQL Cluster Programs".

Additional information about using ndb_mgm can be found in Section 17.5.2, "Commands in the MySQL Cluster Management Client".

This tool extracts current configuration information for data nodes, SQL nodes, and API nodes from one of a number of sources: a MySQL Cluster management node, or its config.ini or my.cnf file. By default, the management node is the source for the configuration data; to override the default, execute ndb_config with the --config-file or --mycnf option. It is also possible to use a data node as the source by specifying its node ID with --config_from_node=node_id-

ndb_config can also provide an offline dump of all configuration parameters which can be used, along with their default, maximum, and minimum values and other information. The dump can be produced in either text or XML format; for more information, see the discussion of the --configinfo and --xml options later in this section).

You can filter the results by section (DB, SYSTEM, or CONNECTIONS) using one of the options --nodes, --system, or --connections.

The following table includes options that are specific to ndb_config. Additional descriptions follow the table. For options common to most MySQL Cluster programs (including ndb_config), see Section 17.4.24, "Options Common to MySQL Cluster Programs - Options Common to MySQL Cluster Programs".

Examples

A utility having this name was formerly part of an internal automated test framework used in testing and debugging MySQL Cluster. It was deprecated in MySQL Cluster NDB 7.0, and removed from MySQL Cluster distributions provided by Oracle beginning with MySQL Cluster NDB 7.2.1.

ndb_delete_all deletes all rows from the given NDB table. In some cases, this can be much faster than DELETE or even TRUNCATE TABLE.

Usage

ndb_delete_all -c connect_string tbl_name -d db_name

This deletes all rows from the table named tbl_name in the database named db_name. It is exactly equivalent to executing TRUNCATE db_name.tbl_name in MySQL.

The following table includes options that are specific to ndb_delete_all. Additional descriptions follow the table. For options common to most MySQL Cluster programs (including ndb_delete_all), see Section 17.4.24, "Options Common to MySQL Cluster Programs - Options Common to MySQL Cluster Programs".

ndb_desc provides a detailed description of one or more NDB tables.

Usage

ndb_desc -c connect_string tbl_name -d db_name [-p]

Sample Output

MySQL table creation and population statements:

USE test;CREATE TABLE fish ( id INT(11) NOT NULL AUTO_INCREMENT, name VARCHAR(20) NOT NULL, length_mm INT(11) NOT NULL, weight_gm INT(11) NOT NULL, PRIMARY KEY pk (id), UNIQUE KEY uk (name)) ENGINE=NDB;INSERT INTO fish VALUES ('','guppy', 35, 2), ('','tuna', 2500, 150000), ('','shark', 3000, 110000), ('','manta ray', 1500, 50000), ('','grouper', 900, 125000), ('','puffer', 250, 2500);

Output from ndb_desc:

shell> ./ndb_desc -c localhost fish -d test -p-- fish --Version: 2Fragment type: 9K Value: 6Min load factor: 78Max load factor: 80Temporary table: noNumber of attributes: 4Number of primary keys: 1Length of frm data: 311Row Checksum: 1Row GCI: 1SingleUserMode: 0ForceVarPart: 1FragmentCount: 2TableStatus: Retrieved-- Attributes --id Int PRIMARY KEY DISTRIBUTION KEY AT=FIXED ST=MEMORY AUTO_INCRname Varchar(20;latin1_swedish_ci) NOT NULL AT=SHORT_VAR ST=MEMORYlength_mm Int NOT NULL AT=FIXED ST=MEMORYweight_gm Int NOT NULL AT=FIXED ST=MEMORY-- Indexes --PRIMARY KEY(id) - UniqueHashIndexPRIMARY(id) - OrderedIndexuk$unique(name) - UniqueHashIndexuk(name) - OrderedIndex-- Per partition info --Partition  Row count  Commit count  Frag fixed memory ...0  2  2 32768 ...1  4  4 32768 ...... Frag varsized memory  Extent_space  Free extent_space... 32768 0 0... 32768 0 0NDBT_ProgramExit: 0 - OK

Information about multiple tables can be obtained in a single invocation of ndb_desc by using their names, separated by spaces. All of the tables must be in the same database.

The Version column in the output contains the table's schema object version. For information about interpreting this value, see NDB Schema Object Versions.

The Extent_space and Free extent_space columns are applicable only to NDB tables having columns on disk; for tables having only in-memory columns, these columns always contain the value 0.

To illustrate their use, we modify the previous example. First, we must create the necessary Disk Data objects, as shown here:

CREATE LOGFILE GROUP lg_1 ADD UNDOFILE 'undo_1.log' INITIAL_SIZE 16M UNDO_BUFFER_SIZE 2M ENGINE NDB;ALTER LOGFILE GROUP lg_1 ADD UNDOFILE 'undo_2.log' INITIAL_SIZE 12M ENGINE NDB;CREATE TABLESPACE ts_1 ADD DATAFILE 'data_1.dat' USE LOGFILE GROUP lg_1 INITIAL_SIZE 32M ENGINE NDB;ALTER TABLESPACE ts_1 ADD DATAFILE 'data_2.dat' INITIAL_SIZE 48M ENGINE NDB;

(For more information on the statements just shown and the objects created by them, see Section 17.5.12.1, "MySQL Cluster Disk Data Objects", as well as Section 13.1.14, "CREATE LOGFILE GROUP Syntax", and Section 13.1.18, "CREATE TABLESPACE Syntax".)

Now we can create and populate a version of the fish table that stores 2 of its columns on disk (deleting the previous version of the table first, if it already exists):

CREATE TABLE fish ( id INT(11) NOT NULL AUTO_INCREMENT, name VARCHAR(20) NOT NULL, length_mm INT(11) NOT NULL, weight_gm INT(11) NOT NULL, PRIMARY KEY pk (id), UNIQUE KEY uk (name)) TABLESPACE ts_1 STORAGE DISK ENGINE=NDB;INSERT INTO fish VALUES ('','guppy', 35, 2), ('','tuna', 2500, 150000), ('','shark', 3000, 110000), ('','manta ray', 1500, 50000), ('','grouper', 900, 125000), ('','puffer', 250, 2500);

When run against this version of the table, ndb_desc displays the following output:

shell> ./ndb_desc -c localhost fish -d test -p-- fish --Version: 3Fragment type: 9K Value: 6Min load factor: 78Max load factor: 80Temporary table: noNumber of attributes: 4Number of primary keys: 1Length of frm data: 321Row Checksum: 1Row GCI: 1SingleUserMode: 0ForceVarPart: 1FragmentCount: 2TableStatus: Retrieved-- Attributes --id Int PRIMARY KEY DISTRIBUTION KEY AT=FIXED ST=MEMORY AUTO_INCRname Varchar(20;latin1_swedish_ci) NOT NULL AT=SHORT_VAR ST=MEMORYlength_mm Int NOT NULL AT=FIXED ST=DISKweight_gm Int NOT NULL AT=FIXED ST=DISK-- Indexes --PRIMARY KEY(id) - UniqueHashIndexPRIMARY(id) - OrderedIndexuk$unique(name) - UniqueHashIndexuk(name) - OrderedIndex-- Per partition info --Partition  Row count  Commit count  Frag fixed memory ...0  2  2 32768 ...1  4  4 32768 ...... Frag varsized memory  Extent_space  Free extent_space... 32768 0 0... 32768 0 0NDBT_ProgramExit: 0 - OK

This means that 1048576 bytes are allocated from the tablespace for this table on each partition, of which 1044440 bytes remain free for additional storage. In other words, 1048576 - 1044440 = 4136 bytes per partition is currently being used to store the data from this table's disk-based columns. The number of bytes shown as Free extent_space is available for storing on-disk column data from the fish table only; for this reason, it is not visible when selecting from the INFORMATION_SCHEMA.FILES table.

The following table includes options that are specific to ndb_desc. Additional descriptions follow the table. For options common to most MySQL Cluster programs (including ndb_desc), see Section 17.4.24, "Options Common to MySQL Cluster Programs - Options Common to MySQL Cluster Programs".

ndb_drop_index drops the specified index from an NDB table. It is recommended that you use this utility only as an example for writing NDB API applications-see the Warning later in this section for details.

Usage

ndb_drop_index -c connect_string table_name index -d db_name

The statement shown above drops the index named index from the table in the database.

The following table includes options that are specific to ndb_drop_index. Additional descriptions follow the table. For options common to most MySQL Cluster programs (including ndb_drop_index), see Section 17.4.24, "Options Common to MySQL Cluster Programs - Options Common to MySQL Cluster Programs".

shell> ./ndb_drop_index -c localhost dogs ix -d ctest1Dropping index dogs/idx...OKNDBT_ProgramExit: 0 - OKshell> ./mysql -u jon -p ctest1Enter password: *******Reading table information for completion of table and column namesYou can turn off this feature to get a quicker startup with -AWelcome to the MySQL monitor.  Commands end with ; or \g.Your MySQL connection id is 7 to server version: 5.5.29-ndb-7.2.10Type 'help;' or '\h' for help. Type '\c' to clear the buffer.mysql> SHOW TABLES;+------------------+| Tables_in_ctest1 |+------------------+| a || bt1  || bt2  || dogs || employees || fish |+------------------+6 rows in set (0.00 sec)mysql> SELECT * FROM dogs;ERROR 1296 (HY000): Got error 4243 'Index not found' from NDBCLUSTER

In such a case, your only option for making the table available to MySQL again is to drop the table and re-create it. You can use either the SQL statementDROP TABLE or the ndb_drop_table utility (see Section 17.4.11, "ndb_drop_table - Drop an NDB Table") to drop the table.

ndb_drop_table drops the specified NDB table. (If you try to use this on a table created with a storage engine other than NDB, the attempt fails with the error 723: No such table exists.) This operation is extremely fast; in some cases, it can be an order of magnitude faster than using a MySQL DROP TABLE statement on an NDB table.

Usage

ndb_drop_table -c connect_string tbl_name -d db_name

The following table includes options that are specific to ndb_drop_table. Additional descriptions follow the table. For options common to most MySQL Cluster programs (including ndb_drop_table), see Section 17.4.24, "Options Common to MySQL Cluster Programs - Options Common to MySQL Cluster Programs".

ndb_error_reporter creates an archive from data node and management node log files that can be used to help diagnose bugs or other problems with a cluster. It is highly recommended that you make use of this utility when filing reports of bugs in MySQL Cluster.

The following table includes command options specific to the MySQL Cluster program ndb_error_reporter. Additional descriptions follow the table. For options common to most MySQL Cluster programs (including ndb_error_reporter), see Section 17.4.24, "Options Common to MySQL Cluster Programs - Options Common to MySQL Cluster Programs".

Usage

ndb_error_reporter path/to/config-file [username] [--fs]

This utility is intended for use on a management node host, and requires the path to the management host configuration file (config.ini). Optionally, you can supply the name of a user that is able to access the cluster's data nodes using SSH, to copy the data node log files. ndb_error_reporter then includes all of these files in archive that is created in the same directory in which it is run. The archive is named ndb_error_report_YYYYMMDDHHMMSS.tar.bz2, where YYYYMMDDHHMMSS is a datetime string.

If the --fs is used, then the data node file systems are also copied to the management host and included in the archive that is produced by this script. As data node file systems can be extremely large even after being compressed, we ask that you please do not send archives created using this option to Oracle unless you are specifically requested to do so.

ndb_index_stat ...

Usage

To obtain basic index statistics about a given NDB table, invoke ndb_index_stat as shown here, with the name of the table as the first argument and the name of the database containing this table specified immediately following it, using the --database (-d) option:

ndb_index_stat table -d database

In this example, we use ndb_index_stat to obtain such information about an NDB table named mytable in the test database:

shell> ndb_index_stat -d test mytabletable:mytable index:PRIMARY fragCount:4sampleVersion:2 loadTime:1336751773 sampleCount:0 keyBytes:0query cache: valid:1 sampleCount:0 totalBytes:0times in ms: save: 12.380 sort: 0.001NDBT_ProgramExit: 0 - OK

The --verbose option provides some additional output, as shown here:

shell> ndb_index_stat -d test mytable --verboserandom seed 1337010518connectedloop 1 of 1table:mytable index:PRIMARY fragCount:4sampleVersion:2 loadTime:1336751773 sampleCount:0 keyBytes:0read statsquery cache createdquery cache: valid:1 sampleCount:0 totalBytes:0times in ms: save: 20.766 sort: 0.001disconnectedNDBT_ProgramExit: 0 - OKshell>

Options

The following table includes options that are specific to the MySQL Cluster ndb_index_stat utility. Additional descriptions are listed following the table. For options common to most MySQL Cluster programs (including ndb_index_stat), see Section 17.4.24, "Options Common to MySQL Cluster Programs - Options Common to MySQL Cluster Programs".

ndb_index_stat statistics options. The following options are used to generate index statistics. They work with a given table and database. They cannot be mixed with ndb_index_stat system options.

ndb_index_stat system options. The following options are used to generate and update the statistics tables in the NDB kernel. They cannot be mixed with ndb_index_stat statistics options.

ndb_print_backup_file obtains diagnostic information from a cluster backup file.

Usage

ndb_print_backup_file file_name

file_name is the name of a cluster backup file. This can be any of the files (.Data, .ctl, or .log file) found in a cluster backup directory. These files are found in the data node's backup directory under the subdirectory BACKUP-#, where # is the sequence number for the backup. For more information about cluster backup files and their contents, see Section 17.5.3.1, "MySQL Cluster Backup Concepts".

Like ndb_print_schema_file and ndb_print_sys_file (and unlike most of the other NDB utilities that are intended to be run on a management server host or to connect to a management server) ndb_print_backup_file must be run on a cluster data node, since it accesses the data node file system directly. Because it does not make use of the management server, this utility can be used when the management server is not running, and even when the cluster has been completely shut down.

Additional Options

None.

ndb_print_schema_file obtains diagnostic information from a cluster schema file.

Usage

ndb_print_schema_file file_name

file_name is the name of a cluster schema file. For more information about cluster schema files, see MySQL Cluster Data Node FileSystemDir Files.

Like ndb_print_backup_file and ndb_print_sys_file (and unlike most of the other NDB utilities that are intended to be run on a management server host or to connect to a management server) ndb_schema_backup_file must be run on a cluster data node, since it accesses the data node file system directly. Because it does not make use of the management server, this utility can be used when the management server is not running, and even when the cluster has been completely shut down.

Additional Options

None.

ndb_print_sys_file obtains diagnostic information from a MySQL Cluster system file.

Usage

ndb_print_sys_file file_name

file_name is the name of a cluster system file (sysfile). Cluster system files are located in a data node's data directory (DataDir); the path under this directory to system files matches the pattern ndb_#_fs/D#/DBDIH/P#.sysfile. In each case, the # represents a number (not necessarily the same number). For more information, see MySQL Cluster Data Node FileSystemDir Files.

Like ndb_print_backup_file and ndb_print_schema_file (and unlike most of the other NDB utilities that are intended to be run on a management server host or to connect to a management server) ndb_print_backup_file must be run on a cluster data node, since it accesses the data node file system directly. Because it does not make use of the management server, this utility can be used when the management server is not running, and even when the cluster has been completely shut down.

Additional Options

None.

Reads a redo log file, checking it for errors, printing its contents in a human-readable format, or both. ndbd_redo_log_reader is intended for use primarily by MySQL Cluster developers and Support personnel in debugging and diagnosing problems.

This utility remains under development, and its syntax and behavior are subject to change in future MySQL Cluster releases.

The C++ source files for ndbd_redo_log_reader can be found in the directory /storage/ndb/src/kernel/blocks/dblqh/redoLogReader.

The following table includes options that are specific to the MySQL Cluster program ndbd_redo_log_reader. Additional descriptions follow the table. For options common to most MySQL Cluster programs (including ndbd_redo_log_reader), see Section 17.4.24, "Options Common to MySQL Cluster Programs - Options Common to MySQL Cluster Programs".

Usage

ndbd_redo_log_reader file_name [options]

file_name is the name of a cluster redo log file. redo log files are located in the numbered directories under the data node's data directory (DataDir); the path under this directory to the redo log files matches the pattern ndb_#_fs/D#/LCP/#/T#F#.Data. In each case, the # represents a number (not necessarily the same number). For more information, see MySQL Cluster Data Node FileSystemDir Files.

The name of the file to be read may be followed by one or more of the options listed here:

Like ndb_print_backup_file and ndb_print_schema_file (and unlike most of the NDB utilities that are intended to be run on a management server host or to connect to a management server) ndbd_redo_log_reader must be run on a cluster data node, since it accesses the data node file system directly. Because it does not make use of the management server, this utility can be used when the management server is not running, and even when the cluster has been completely shut down.

The cluster restoration program is implemented as a separate command-line utility ndb_restore, which can normally be found in the MySQL bin directory. This program reads the files created as a result of the backup and inserts the stored information into the database.

ndb_restore must be executed once for each of the backup files that were created by the START BACKUP command used to create the backup (see Section 17.5.3.2, "Using The MySQL Cluster Management Client to Create a Backup"). This is equal to the number of data nodes in the cluster at the time that the backup was created.

The following table includes options that are specific to the MySQL Cluster native backup restoration program ndb_restore. Additional descriptions follow the table. For options common to most MySQL Cluster programs (including ndb_restore), see Section 17.4.24, "Options Common to MySQL Cluster Programs - Options Common to MySQL Cluster Programs".

Typical options for this utility are shown here:

ndb_restore [-c connectstring] -n node_id [-m] -b backup_id \ -r --backup_path=/path/to/backup/files

The -c option is used to specify a connectstring which tells ndb_restore where to locate the cluster management server. (See Section 17.3.2.3, "The MySQL Cluster Connectstring", for information on connectstrings.) If this option is not used, then ndb_restore attempts to connect to a management server on localhost:1186. This utility acts as a cluster API node, and so requires a free connection "slot" to connect to the cluster management server. This means that there must be at least one [api] or [mysqld] section that can be used by it in the cluster config.ini file. It is a good idea to keep at least one empty [api] or [mysqld] section in config.ini that is not being used for a MySQL server or other application for this reason (see Section 17.3.2.7, "Defining SQL and Other API Nodes in a MySQL Cluster").

You can verify that ndb_restore is connected to the cluster by using the SHOW command in the ndb_mgm management client. You can also accomplish this from a system shell, as shown here:

shell> ndb_mgm -e "SHOW"

-n is used to specify the node ID of the data node on which the backups were taken.

The first time you run the ndb_restore restoration program, you also need to restore the metadata. In other words, you must re-create the database tables-this can be done by running it with the -m option. Note that the cluster should have an empty database when starting to restore a backup. (In other words, you should start ndbd with --initial prior to performing the restore. You should also remove manually any Disk Data files present in the data node's DataDir.)

It is possible to restore data without restoring table metadata. The default behavior when doing this is for ndb_restore to fail with an error if table data do not match the table schema; this can be overridden using the --skip-table-check or -s option.

Some of the restrictions on mismatches in column definitions when restoring data using ndb_restore are relaxed; when one of these types of mismatches is encountered, ndb_restore does not stop with an error as it did previously, but rather accepts the data and inserts it into the target table while issuing a warning to the user that this is being done. This behavior occurs whether or not either of the options --skip-table-check or --promote-attributes is in use. These differences in column definitions are of the following types:

ndb_restore supports limited attribute promotion in much the same way that it is supported by MySQL replication; that is, data backed up from a column of a given type can generally be restored to a column using a "larger, similar" type. For example, data from a CHAR(20) column can be restored to a column declared as VARCHAR(20), VARCHAR(30), or CHAR(30); data from a MEDIUMINT column can be restored to a column of type INT or BIGINT. See Section 16.4.1.8.2, "Replication of Columns Having Different Data Types", for a table of type conversions currently supported by attribute promotion.

Attribute promotion by ndb_restore must be enabled explicitly, as follows:

--lossy-conversions, -L

This option is intended to complement the --promote-attributes option. Using --lossy-conversions allows lossy conversions of column values (type demotions or changes in sign) when restoring data from backup. With some exceptions, the rules governing demotion are the same as for MySQL replication; see Section 16.4.1.8.2, "Replication of Columns Having Different Data Types", for information about specific type conversions currently supported by attribute demotion.

ndb_restore reports any truncation of data that it performs during lossy conversions once per attribute and column.

The --preserve-trailing-spaces option (short form -R) causes trailing spaces to be preserved when promoting a fixed-width character data type to its variable-width equivalent-that is, when promoting a CHAR column value to VARCHAR or a BINARY column value to VARBINARY. Otherwise, any trailing spaces are dropped from such column values when they are inserted into the new columns.

The -b option is used to specify the ID or sequence number of the backup, and is the same number shown by the management client in the Backup backup_id completed message displayed upon completion of a backup. (See Section 17.5.3.2, "Using The MySQL Cluster Management Client to Create a Backup".)

--restore_epoch (short form: -e) adds (or restores) epoch information to the cluster replication status table. This is useful for starting replication on a MySQL Cluster replication slave. When this option is used, the row in the mysql.ndb_apply_status having 0 in the id column is updated if it already exists; such a row is inserted if it does not already exist. (See Section 17.6.9, "MySQL Cluster Backups With MySQL Cluster Replication".)

--restore_data

This option causes ndb_restore to output NDB table data and logs.

--restore_meta

This option causes ndb_restore to print NDB table metadata.

--restore-privilege-tables

ndb_restore does not by default restore distributed MySQL privilege tables (MySQL Cluster NDB 7.2.0 and later). This option causes ndb_restore to restore the privilege tables.

This works only if the privilege tables were converted to NDB before the backup was taken. For more information, see Section 17.5.14, "Distributed MySQL Privileges for MySQL Cluster".

--backup_path

The path to the backup directory is required; this is supplied to ndb_restore using the --backup_path option, and must include the subdirectory corresponding to the ID backup of the backup to be restored. For example, if the data node's DataDir is /var/lib/mysql-cluster, then the backup directory is /var/lib/mysql-cluster/BACKUP, and the backup files for the backup with the ID 3 can be found in /var/lib/mysql-cluster/BACKUP/BACKUP-3. The path may be absolute or relative to the directory in which the ndb_restore executable is located, and may be optionally prefixed with backup_path=.

It is possible to restore a backup to a database with a different configuration than it was created from. For example, suppose that a backup with backup ID 12, created in a cluster with two database nodes having the node IDs 2 and 3, is to be restored to a cluster with four nodes. Then ndb_restore must be run twice-once for each database node in the cluster where the backup was taken. However, ndb_restore cannot always restore backups made from a cluster running one version of MySQL to a cluster running a different MySQL version. See Section 17.2.7, "Upgrading and Downgrading MySQL Cluster NDB 7.2", for more information.

For more rapid restoration, the data may be restored in parallel, provided that there is a sufficient number of cluster connections available. That is, when restoring to multiple nodes in parallel, you must have an [api] or [mysqld] section in the cluster config.ini file available for each concurrent ndb_restore process. However, the data files must always be applied before the logs.

--no-upgrade

When using ndb_restore to restore a backup, VARCHAR columns created using the old fixed format are resized and recreated using the variable-width format now employed. This behavior can be overridden using the --no-upgrade option (short form: -u) when running ndb_restore.

--print_data

The --print_data option causes ndb_restore to direct its output to stdout.

TEXT and BLOB column values are always truncated to the first 256 bytes in the output; this cannot currently be overridden when using --print_data.

Several additional options are available for use with the --print_data option in generating data dumps, either to stdout, or to a file. These are similar to some of the options used with mysqldump, and are shown in the following list:

--print_meta

This option causes ndb_restore to print all metadata to stdout.

--print_log

The --print_log option causes ndb_restore to output its log to stdout.

--print

Causes ndb_restore to print all data, metadata, and logs to stdout. Equivalent to using the --print_data, --print_meta, and --print_log options together.

--dont_ignore_systab_0

Normally, when restoring table data and metadata, ndb_restore ignores the copy of the NDB system table that is present in the backup. --dont_ignore_systab_0 causes the system table to be restored. This option is intended for experimental and development use only, and is not recommended in a production environment.

--ndb-nodegroup-map, -z

This option can be used to restore a backup taken from one node group to a different node group. Its argument is a list of the form source_node_group, target_node_group.

--no-binlog

This option prevents any connected SQL nodes from writing data restored by ndb_restore to their binary logs.

--no-restore-disk-objects, -d

This option stops ndb_restore from restoring any MySQL Cluster Disk Data objects, such as tablespaces and log file groups; see Section 17.5.12, "MySQL Cluster Disk Data Tables", for more information about these.

--parallelism=#, -p

Determines the maximum number of parallel transactions that ndb_restore tries to use. By default, this is 128; the minimum is 1, and the maximum is 1024.

--progress-frequency=N

Print a status report each N seconds while the backup is in progress. 0 (the default) causes no status reports to be printed. The maximum is 65535.

--verbose=#

Sets the level for the verbosity of the output. The minimum is 0; the maximum is 255. The default value is 1.

It is possible to restore only selected databases, or selected tables from a single database, using the syntax shown here:

ndb_restore other_options db_name,[db_name[,...] | tbl_name[,tbl_name][,...]]

In other words, you can specify either of the following to be restored:

--include-databases=db_name[,db_name][,...]

--include-tables=db_name.tbl_name[,db_name.tbl_name][,...]

Use the --include-databases option or the --include-tables option for restoring only specific databases or tables, respectively. --include-databases takes a comma-delimited list of databases to be restored. --include-tables takes a comma-delimited list of tables (in database.table format) to be restored.

When --include-databases or --include-tables is used, only those databases or tables named by the option are restored; all other databases and tables are excluded by ndb_restore, and are not restored.

The following table shows several invocations of ndb_restore using --include-* options (other options possibly required have been omitted for clarity), and the effects these have on restoring from a MySQL Cluster backup:

You can also use these two options together. For example, the following causes all tables in databases db1 and db2, together with the tables t1 and t2 in database db3, to be restored (and no other databases or tables):

shell> ndb_restore [...] --include-databases=db1,db2 --include-tables=db3.t1,db3.t2

(Again we have omitted other, possibly required, options in the example just shown.)

--exclude-databases=db_name[,db_name][,...]

--exclude-tables=db_name.tbl_name[,db_name.tbl_name][,...]

It is possible to prevent one or more databases or tables from being restored using the ndb_restore options --exclude-databases and --exclude-tables. --exclude-databases takes a comma-delimited list of one or more databases which should not be restored. --exclude-tables takes a comma-delimited list of one or more tables (using database.table format) which should not be restored.

When --exclude-databases or --exclude-tables is used, only those databases or tables named by the option are excluded; all other databases and tables are restored by ndb_restore.

This table shows several invocations of ndb_restore usng --exclude-* options (other options possibly required have been omitted for clarity), and the effects these options have on restoring from a MySQL Cluster backup:

You can use these two options together. For example, the following causes all tables in all databases except for databases db1 and db2, along with the tables t1 and t2 in database db3, not to be restored:

shell> ndb_restore [...] --exclude-databases=db1,db2 --exclude-tables=db3.t1,db3.t2

(Again, we have omitted other possibly necessary options in the interest of clarity and brevity from the example just shown.)

You can use --include-* and --exclude-* options together, subject to the following rules:

For example, the following set of options causes ndb_restore to restore all tables from database db1 except db1.t1, while restoring no other tables from any other databases:

  --include-databases=db1 --exclude-tables=db1.t1

However, reversing the order of the options just given simply causes all tables from database db1 to be restored (including db1.t1, but no tables from any other database), because the --include-databases option, being farthest to the right, is the first match against database db1 and thus takes precedence over any other option that matches db1 or any tables in db1:

  --exclude-tables=db1.t1 --include-databases=db1

--exclude-missing-columns

It is also possible to restore only selected table columns using the --exclude-missing-columns option. When this option is used, ndb_restore ignores any columns missing from tables being restored as compared to the versions of those tables found in the backup. This option applies to all tables being restored. If you wish to apply this option only to selected tables or databases, you can use it in combination with one or more of the options described in the previous paragraph to do so, then restore data to the remaining tables using a complementary set of these options.

--disable-indexes

Disable restoration of indexes during restoration of the data from a native NDB backup. Afterwards, you can restore indexes for all tables at once with multi-threaded building of indexes using --rebuild-indexes, which should be faster than rebuilding indexes concurrently for very large tables.

--rebuild-indexes

You can use this option with ndb_restore to cause multi-threaded rebuilding of the ordered indexes while restoring a native NDB backup.

--skip-broken-objects

This option causes ndb_restore to ignore corrupt tables while reading a native NDB backup, and to continue restoring any remaining tables (that are not also corrupted). Currently, the --skip-broken-objects option works only in the case of missing blob parts tables.

--skip-unknown-objects

This option causes ndb_restore to ignore any schema objects it does not recognize while reading a native NDB backup. This can be used for restoring a backup made from a cluster running MySQL Cluster NDB 7.2 to a cluster running MySQL Cluster NDB 7.1.

--rewrite-database=old_dbname,new_dbname

This option makes it possible to restore to a database having a different name from that used in the backup. For example, if a backup is made of a database named products, you can restore the data it contains to a database named inventory, use this option as shown here (omitting any other options that might be required):

shell> ndb_restore --rewrite-database=product,inventory

The option can be employed multiple times in a single invocation of ndb_restore. Thus it is possible to restore simultaneously from a database named db1 to a database named db2 and from a database named db3 to one named db4 using --rewrite-database=db1,db2 --rewrite-database=db3,db4. Other ndb_restore options may be used between multiple occurrences of --rewrite-database.

In the event of conflicts between multiple --rewrite-database options, the last --rewrite-database option used, reading from left to right, is the one that takes effect. For example, if --rewrite-database=db1,db2 --rewrite-database=db1,db3 is used, only --rewrite-database=db1,db3 is honored, and --rewrite-database=db1,db2 is ignored. It is also possible to restore from multiple databases to a single database, so that --rewrite-database=db1,db3 --rewrite-database=db2,db3 restores all tables and data from databases db1 and db2 into database db3.

Error reporting. ndb_restore reports both temporary and permanent errors. In the case of temporary errors, it may able to recover from them, and reports Restore successful, but encountered temporary error, please look at configuration in such cases.

ndb_select_all prints all rows from an NDB table to stdout.

Usage

ndb_select_all -c connect_string tbl_name -d db_name [> file_name]

The following table includes options that are specific to the MySQL Cluster native backup restoration program ndb_select_all. Additional descriptions follow the table. For options common to most MySQL Cluster programs (including ndb_select_all), see Section 17.4.24, "Options Common to MySQL Cluster Programs - Options Common to MySQL Cluster Programs".

Sample Output

Output from a MySQL SELECT statement:

mysql> SELECT * FROM ctest1.fish;+----+-----------+| id | name  |+----+-----------+|  3 | shark ||  6 | puffer ||  2 | tuna  ||  4 | manta ray ||  5 | grouper   ||  1 | guppy |+----+-----------+6 rows in set (0.04 sec)

Output from the equivalent invocation of ndb_select_all:

shell> ./ndb_select_all -c localhost fish -d ctest1id  name3   [shark]6   [puffer]2   [tuna]4   [manta ray]5   [grouper]1   [guppy]6 rows returnedNDBT_ProgramExit: 0 - OK

Note that all string values are enclosed by square brackets ("[...]") in the output of ndb_select_all. For a further example, consider the table created and populated as shown here:

CREATE TABLE dogs ( id INT(11) NOT NULL AUTO_INCREMENT, name VARCHAR(25) NOT NULL, breed VARCHAR(50) NOT NULL, PRIMARY KEY pk (id), KEY ix (name))TABLESPACE ts STORAGE DISKENGINE=NDBCLUSTER;INSERT INTO dogs VALUES ('', 'Lassie', 'collie'), ('', 'Scooby-Doo', 'Great Dane'), ('', 'Rin-Tin-Tin', 'Alsatian'), ('', 'Rosscoe', 'Mutt');

This demonstrates the use of several additional ndb_select_all options:

shell> ./ndb_select_all -d ctest1 dogs -o ix -z --gci --diskGCI id name  breed DISK_REF834461  2  [Scooby-Doo]  [Great Dane] [ m_file_no: 0 m_page: 98 m_page_idx: 0 ]834878  4  [Rosscoe] [Mutt]   [ m_file_no: 0 m_page: 98 m_page_idx: 16 ]834463  3  [Rin-Tin-Tin] [Alsatian]   [ m_file_no: 0 m_page: 34 m_page_idx: 0 ]835657  1  [Lassie]  [Collie] [ m_file_no: 0 m_page: 66 m_page_idx: 0 ]4 rows returnedNDBT_ProgramExit: 0 - OK

ndb_select_count prints the number of rows in one or more NDB tables. With a single table, the result is equivalent to that obtained by using the MySQL statement SELECT COUNT(*) FROM tbl_name.

Usage

ndb_select_count [-c connect_string] -ddb_name tbl_name[, tbl_name2[, ...]]

The following table includes options that are specific to the MySQL Cluster native backup restoration program ndb_select_count. Additional descriptions follow the table. For options common to most MySQL Cluster programs (including ndb_select_count), see Section 17.4.24, "Options Common to MySQL Cluster Programs - Options Common to MySQL Cluster Programs".

You can obtain row counts from multiple tables in the same database by listing the table names separated by spaces when invoking this command, as shown under Sample Output.

Sample Output

shell> ./ndb_select_count -c localhost -d ctest1 fish dogs6 records in table fish4 records in table dogsNDBT_ProgramExit: 0 - OK

ndb_show_tables displays a list of all NDB database objects in the cluster. By default, this includes not only both user-created tables and NDB system tables, but NDB-specific indexes, internal triggers, and MySQL Cluster Disk Data objects as well.

The following table includes options that are specific to the MySQL Cluster native backup restoration program ndb_show_tables. Additional descriptions follow the table. For options common to most MySQL Cluster programs (including ndb_show_tables), see Section 17.4.24, "Options Common to MySQL Cluster Programs - Options Common to MySQL Cluster Programs".

Usage

ndb_show_tables [-c connect_string]

This is a Perl script that can be used to estimate the amount of space that would be required by a MySQL database if it were converted to use the NDBCLUSTER storage engine. Unlike the other utilities discussed in this section, it does not require access to a MySQL Cluster (in fact, there is no reason for it to do so). However, it does need to access the MySQL server on which the database to be tested resides.

Requirements

ndb_size.pl can also be found in the MySQL sources in storage/ndb/tools.

The following table includes options that are specific to the MySQL Cluster program ndb_size.pl. Additional descriptions follow the table. For options common to most MySQL Cluster programs (including ndb_size.pl), see Section 17.4.24, "Options Common to MySQL Cluster Programs - Options Common to MySQL Cluster Programs".

Usage

perl ndb_size.pl [--database={db_name|ALL}] [--hostname=host[:port]] [--socket=socket] \  [--user=user] [--password=password]  \   [--help|-h] [--format={html|text}] \   [--loadqueries=file_name] [--savequeries=file_name]

By default, this utility attempts to analyze all databases on the server. You can specify a single database using the --database option; the default behavior can be made explicit by using ALL for the name of the database. You can also exclude one or more databases by using the --excludedbs option with a comma-separated list of the names of the databases to be skipped. Similarly, you can cause specific tables to be skipped by listing their names, separated by commas, following the optional --excludetables option. A host name can be specified using --hostname; the default is localhost. In MySQL Cluster NDB 7.2.6 and later, you can specify a port in addition to the host using host:port format for the value of --hostname. The default port number is 3306. If necessary, you can also specify a socket; the default is /var/lib/mysql.sock. A MySQL user name and password can be specified the corresponding options shown. It also possible to control the format of the output using the --format option; this can take either of the values html or text, with text being the default. An example of the text output is shown here:

shell> ndb_size.pl --database=test --socket=/tmp/mysql.sockndb_size.pl report for database: 'test' (1 tables)--------------------------------------------------Connected to: DBI:mysql:host=localhost;mysql_socket=/tmp/mysql.sockIncluding information for versions: 4.1, 5.0, 5.1test.t1-------DataMemory for Columns (* means varsized DataMemory): Column Name Type  Varsized   Key  4.1  5.0   5.1 HIDDEN_NDB_PKEY  bigint PRI 8 8 8  c2 varchar(50) Y 52   52 4*  c1 int(11) 4 4 4   --   -- --Fixed Size Columns DM/Row  64   64 12   Varsize Columns DM/Row   0 0 4DataMemory for Indexes:   Index Name Type 4.1 5.0 5.1  PRIMARY BTREE 16 16 16   -- -- --   Total Index DM/Row  16 16 16IndexMemory for Indexes:   Index Name 4.1 5.0 5.1  PRIMARY 33 16 16  -- -- --   Indexes IM/Row 33 16 16Summary (for THIS table): 4.1 5.0 5.1 Fixed Overhead DM/Row 12 12 16   NULL Bytes/Row  4  4  4   DataMemory/Row 96 96 48  (Includes overhead, bitmap and indexes)  Varsize Overhead DM/Row  0  0  8   Varsize NULL Bytes/Row  0  0  4   Avg Varside DM/Row  0  0 16 No. Rows  0  0  0 Rows/32kb DM Page 340 340 680Fixedsize DataMemory (KB)  0  0  0Rows/32kb Varsize DM Page  0  0   2040  Varsize DataMemory (KB)  0  0  0 Rows/8kb IM Page 248 512 512 IndexMemory (KB)  0  0  0Parameter Minimum Requirements------------------------------* indicates greater than default Parameter Default 4.1 5.0 5.1  DataMemory (KB)   81920  0   0   0   NoOfOrderedIndexes 128  1   1   1   NoOfTables 128  1   1   1 IndexMemory (KB)   18432  0   0   0 NoOfUniqueHashIndexes  64  0   0   0   NoOfAttributes 1000  3   3   3 NoOfTriggers 768  5   5   5

For debugging purposes, the Perl arrays containing the queries run by this script can be read from the file specified using can be saved to a file using --savequeries; a file containing such arrays to be read in during script execution can be specified using --loadqueries. Neither of these options has a default value.

To produce output in HTML format, use the --format option and redirect the output to a file, as shown here:

shell> ndb_size.pl --database=test --socket=/tmp/mysql.sock --format=html > ndb_size.html

(Without the redirection, the output is sent to stdout.)

The output from this script includes the following information:

ndb_waiter repeatedly (each 100 milliseconds) prints out the status of all cluster data nodes until either the cluster reaches a given status or the --timeout limit is exceeded, then exits. By default, it waits for the cluster to achieve STARTED status, in which all nodes have started and connected to the cluster. This can be overridden using the --no-contact and --not-started options.

The node states reported by this utility are as follows:

The following table includes options that are specific to the MySQL Cluster native backup restoration program ndb_waiter. Additional descriptions follow the table. For options common to most MySQL Cluster programs (including ndb_waiter), see Section 17.4.24, "Options Common to MySQL Cluster Programs - Options Common to MySQL Cluster Programs".

Usage

ndb_waiter [-c connect_string]

Additional Options

Sample Output. Shown here is the output from ndb_waiter when run against a 4-node cluster in which two nodes have been shut down and then started again manually. Duplicate reports (indicated by "...") are omitted.

shell> ./ndb_waiter -c localhostConnecting to mgmsrv at (localhost)State node 1 STARTEDState node 2 NO_CONTACTState node 3 STARTEDState node 4 NO_CONTACTWaiting for cluster enter state STARTED...State node 1 STARTEDState node 2 UNKNOWNState node 3 STARTEDState node 4 NO_CONTACTWaiting for cluster enter state STARTED...State node 1 STARTEDState node 2 STARTINGState node 3 STARTEDState node 4 NO_CONTACTWaiting for cluster enter state STARTED...State node 1 STARTEDState node 2 STARTINGState node 3 STARTEDState node 4 UNKNOWNWaiting for cluster enter state STARTED...State node 1 STARTEDState node 2 STARTINGState node 3 STARTEDState node 4 STARTINGWaiting for cluster enter state STARTED...State node 1 STARTEDState node 2 STARTEDState node 3 STARTEDState node 4 STARTINGWaiting for cluster enter state STARTED...State node 1 STARTEDState node 2 STARTEDState node 3 STARTEDState node 4 STARTEDWaiting for cluster enter state STARTEDNDBT_ProgramExit: 0 - OK

All MySQL Cluster programs accept the options described in this section, with the following exceptions:

Users of earlier MySQL Cluster versions should note that some of these options have been changed to make them consistent with one another as well as with mysqld. You can use the --help option with any MySQL Cluster program-with the exception of ndb_print_backup_file, ndb_print_schema_file, and ndb_print_sys_file-to view a list of the options which the program supports.

The options in the following table are common to all MySQL Cluster executables (except those noted previously in this section).

For options specific to individual MySQL Cluster programs, see Section 17.4, "MySQL Cluster Programs".

See Section 17.3.4.2, "MySQL Server Options for MySQL Cluster", for mysqld options relating to MySQL Cluster.