Web lists-archives.com

MySQL Cluster 7.5.2 has been released (part 1/2)




Dear MySQL Users,

MySQL Cluster is the distributed, shared-nothing variant of MySQL.
This storage engine provides:

  - In-Memory storage - Real-time performance (with optional
    checkpointing to disk)
  - Transparent Auto-Sharding - Read & write scalability
  - Active-Active/Multi-Master geographic replication

  - 99.999% High Availability with no single point of failure
    and on-line maintenance
  - NoSQL and SQL APIs (including C++, Java, http, Memcached
    and JavaScript/Node.js)

MySQL Cluster 7.5.2 DMR, has been released and can be downloaded from

http://www.mysql.com/downloads/cluster/

where you will also find Quick Start guides to help you get your
first MySQL Cluster database up and running.

The release notes are available from

http://dev.mysql.com/doc/relnotes/mysql-cluster/7.5/en/index.html

MySQL Cluster enables users to meet the database challenges of next
generation web, cloud, and communications services with uncompromising
scalability, uptime and agility.

More details can be found at

http://www.mysql.com/products/cluster/

Enjoy !

Changes in MySQL Cluster NDB 7.5.2 (5.7.12-ndb-7.5.2) (2016-06-01,
Developer Milestone 3)

   MySQL Cluster NDB 7.5.2 is a new release of MySQL Cluster
   7.5, based on MySQL Server 5.7 and including features in
   version 7.5 of the NDB
   (http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster.html)
   storage engine, as well as fixing recently discovered bugs in
   previous MySQL Cluster releases.

   Obtaining MySQL Cluster NDB 7.5.  MySQL Cluster NDB 7.5
   source code and binaries can be obtained from
http://dev.mysql.com/downloads/cluster/.

   For an overview of changes made in MySQL Cluster NDB 7.5, see
   What is New in MySQL Cluster NDB 7.5
   (http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-what-is
-new.html <http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-what-is-new.html>).

   This release also incorporates all bugfixes and changes made
   in previous MySQL Cluster releases, as well as all bugfixes
   and feature changes which were added in mainline MySQL 5.7
   through MySQL 5.7.12 (see Changes in MySQL 5.7.12
   (2016-04-11)
   (http://dev.mysql.com/doc/relnotes/mysql/5.7/en/news-5-7-12.h
tml <http://dev.mysql.com/doc/relnotes/mysql/5.7/en/news-5-7-12.html>)).

   Functionality Added or Changed

     * Performance: A deficiency in event buffer memory
       allocation was identified as inefficient and possibly
       leading to undesirable results. This could happen when
       additional memory was allocated from the operating system
       to buffer received event data even when memory had
       already been allocated but remained unused. This is fixed
       by allocating the event buffer memory directly from the
       page allocation memory manager (mmap()), where such
       functionality is offered by the operating system,
       allowing for direct control over the memory such that it
       is in fact returned to the system when released.
       This remimplementation avoids the tendencies of the
       existing one to approach worst-case memory usage,
       maintainence of data structures for a worst-case event
       buffer event count, and useless caching of free memory in
       unusable positions. This work should also help minimize
       the runtime costs of buffering events, minimize heap
       fragmentation, and avoid OS-specific problems due to
       excessive numbers of distinct memory mappings.
       In addition, the relationship between epochs and internal
       EventData objects is now preserved throughout the event
       lifecycle, reception to consumption, thus removing the
       need for iterating, and keeping in synch, two different
       lists representing the epochs and their EventData
       objects.
       As part of this work, better reporting on the relevant
       event buffer metrics is now provided in the cluster logs.
       References: See also: Bug #21651536, Bug #21660947, Bug
       #21661297, Bug #21673318, Bug #21689380, Bug #21809959.

     * ndb_restore now performs output logging for specific
       stages of its operation. (Bug #21097957)

     * An improvement in the hash index implementation used by
       MySQL Cluster data nodes means that partitions may now
       contain more than 16 GB of data for fixed columns, and
       the maximum partition size for fixed column data is now
       increased to 128 TB. The previous limitation originated
       with the DBACC block in the NDB
       (http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster.htm
l <http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster.html>) kernel using only 32-bit references to the fixed-size
       part of a row handled in the DBTUP block, even though
       45-bit references were already in use elsewhere in the
       kernel outside the DBACC block; all such references in
       DBACC now use 45-bit pointers instead.
       As part of this work, error messages returned by the
       DBACC kernel block that were overly generic have now been
       improved by making them more specific. (Bug #17465232)

     * The NDB
       (http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster.htm
l <http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster.html>) storage engine supports the MySQL JSON
       (http://dev.mysql.com/doc/refman/5.7/en/json.html) data
       type and MySQL JSON functions implemented in MySQL 5.7.8
       and later. This support is subject to the limitation that
       a single NDB table can have at most 3 JSON columns.

     * A number of changes and improvements were made to the
       handling of send threads by NDB
       (http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster.htm
l <http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster.html>). The changes are summarized with brief descriptions in
       the following list:

          + Decreased resource requirements for send threads,
            making sure that a given configuration using send
            threads outperforms the same configuration without
            send threads.

          + Made use of otherwise idle threads (other than
            receiver threads) as send threads without incurring
            extra CPU resources in low-load situations.

          + Improved response time for write transactions.

          + Provided for handling of bad configuration data in a
            more graceful manner.

          + Made it possible to measure CPU usage with improved
            real-time reporting from data nodes on their
            resource usage.
       As part of implementing the last item of those just
       listed, a number of new tables providing information
       about CPU and thread activity by node, thread ID, and
       thread type have been added to the ndbinfo
       (http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-ndb
info.html <http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-ndbinfo.html>) information database. These tables are listed
       here:

          + cpustat
            (http://dev.mysql.com/doc/refman/5.7/en/mysql-cluste
r-ndbinfo-cpustat.html <http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-ndbinfo-cpustat.html>): Provides per-second,
            per-thread CPU statistics

          + cpustat_50ms
            (http://dev.mysql.com/doc/refman/5.7/en/mysql-cluste
r-ndbinfo-cpustat-50ms.html <http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-ndbinfo-cpustat-50ms.html>): Shows raw per-thread
            CPU statistics data, gathered every 50ms

          + cpustat_1sec
            (http://dev.mysql.com/doc/refman/5.7/en/mysql-cluste
r-ndbinfo-cpustat-1sec.html <http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-ndbinfo-cpustat-1sec.html>): Provides raw
            per-thread CPU statistics data, gathered each second

          + cpustat_20sec
            (http://dev.mysql.com/doc/refman/5.7/en/mysql-cluste
r-ndbinfo-cpustat-20sec.html <http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-ndbinfo-cpustat-20sec.html>): Displays raw
            per-thread CPU statistics data, gathered every 20
            seconds

          + threads
            (http://dev.mysql.com/doc/refman/5.7/en/mysql-cluste
r-ndbinfo-threads.html <http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-ndbinfo-threads.html>): Shows names and
            descriptions of thread types
       For more information, see The ndbinfo MySQL Cluster
       Information Database
       (http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-ndb
info.html <http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-ndbinfo.html>).

     * Rewrote the implementation of the NDB
       (http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster.htm
l <http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster.html>) storage engine's global schema lock functionality to
       make use of the metadata lock hooks implemented in the
       MySQL Server in MySQL 5.7.11.

     * A number of improvements provide additional read
       scalability for NDB by making it possible to read tables
       locally. It is now possible to enable reads from any
       replica, rather than from the primary replica only. This
       is disabled by default to remain compatible with previous
       behavior, but can be enabled for a given SQL node using
       the ndb_read_backup
       (http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-opt
ions-variables.html#sysvar_ndb_read_backup <http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-options-variables.html#sysvar_ndb_read_backup>) system
       variable added in this release.
       It also becomes possible to be more flexible about the
       assignment of partitions by setting a fragment count
       type. Possible count types are one per node, one per node
       group, one per Local Data Manager (LDM) per node (this is
       the previous assignment scheme and still the default),
       and one per LDM per node group. This setting can be
       controlled for individual tables by means of a
       FRAGMENT_COUNT_TYPE option embedded in an NDB_TABLE
       comment in CREATE TABLE
       (http://dev.mysql.com/doc/refman/5.7/en/create-table.html
       ) or ALTER TABLE
       (http://dev.mysql.com/doc/refman/5.7/en/alter-table.html)
       .
       Tables using one of the two per-node-group settings for
       the fragment count type can also be fully replicated.
       This requires that the table's fragment count type is
       ONE_PER_NODE_GROUP or ONE_PER_LDM_PER_NODE_GROUP, and can
       be enabled using the option FULLY_REPLICATED=1 within in
       an NDB_TABLE comment. The option can be enabled by
       default for all new NDB tables using the
       ndb_fully_replicated
       (http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-opt
ions-variables.html#sysvar_ndb_fully_replicated <http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-options-variables.html#sysvar_ndb_fully_replicated>) system
       variable added in this release.
       Settings for table-level READ_BACKUP are also supported
       using the COMMENT="NDB_TABLE=..." syntax. It is also
       possible (and often preferable) to set multiple options
       in one comment within a single CREATE TABLE or ALTER
       TABLE statement. For more information and examples, see
       Setting NDB_TABLE options in table comments
       (http://dev.mysql.com/doc/refman/5.7/en/create-table-ndb-
table-comment-options.html <http://dev.mysql.com/doc/refman/5.7/en/create-table-ndb-table-comment-options.html>).
       This release also introduces the ndb_data_node_neighbour
       (http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-opt
ions-variables.html#sysvar_ndb_data_node_neighbour <http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-options-variables.html#sysvar_ndb_data_node_neighbour>)
       system variable, which is intended to be employed with
       transaction hinting (and fully-replicated tables), and to
       provide the current SQL node with the ID of a "nearby"
       data node to use. See the description of this variable in
       the documentation for more information.
       References: See also: Bug #18435416.

     * Made the following enhancements and additions to the
       ThreadConfig multithreaded data node (ndbmtd)
       configuration parameter:

          + Added support for non-exclusive CPU locking on
            FreeBSD and Windows using cpubind and cpuset.

          + Added support for exclusive CPU locking on Solaris,
            using cpubind_exclusive and cpuset_exclusive, which
            are added in this release.

          + Added thread prioritzation using thread_prio, which
            is added in this release. thread_prio is supported
            on Linux, FreeBSD, Windows, and Solaris, but the
            exact effects of this setting are platform-specific;
            see the documentation for details.

          + Added support for setting realtime on Windows
            platforms.
       For more information, see the description of the
       ThreadConfig
       (http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-ndb
d-definition.html#ndbparam-ndbmtd-threadconfig <http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-ndbd-definition.html#ndbparam-ndbmtd-threadconfig>) parameter
       in the online documentation.

     * Cluster API: Added the
       Ndb::setEventBufferQueueEmptyEpoch()
       (http://dev.mysql.com/doc/ndbapi/en/ndb-ndb-methods.html#
ndb-ndb-seteventbufferqueueemptyepoch <http://dev.mysql.com/doc/ndbapi/en/ndb-ndb-methods.html#ndb-ndb-seteventbufferqueueemptyepoch>) method, which
       makes it possible to enable queuing of empty events
       (event type TE_EMPTY). (Bug #22157845)

     * Cluster API: Ndb_free_list_t is a template used in the
       implementation of the NDB API to create a list keeping
       released API objects such as NdbTransaction
       (http://dev.mysql.com/doc/ndbapi/en/ndb-ndbtransaction.ht
ml <http://dev.mysql.com/doc/ndbapi/en/ndb-ndbtransaction.html>), NdbOperation
       (http://dev.mysql.com/doc/ndbapi/en/ndb-ndboperation.html
       ), and NdbRecAttr
       (http://dev.mysql.com/doc/ndbapi/en/ndb-ndbrecattr.html).
       One drawback to this template is that released API
       objects are kept in the list for the lifetime of the
       owning Ndb
       (http://dev.mysql.com/doc/ndbapi/en/ndb-ndb.html) object,
       such that a transient peak in the demand for any object
       causes an effective leak in memory that persists until
       this Ndb object itself has been released.
       This work adds statistics to each Ndb_free_list instance
       which samples the usage of objects maintained by the
       list; now, when objects are released, they can be
       released into the free list, or deallocated, based on the
       collected usage statistics.

   Continued in part 2...