Spectra Logic Developer Program News
      Spectra Developer Program News  
      April 2016
 

Check out the latest news from the Spectra Developer Program. We have been very busy since our last quarterly update!

BlackPearl 3.0 Now Shipping; Updated SDK, Simulator, Documentation

We are happy to announce that Version 3.0 of BlackPearl has been released! It is now shipping on new BlackPearl hardware, and it is available to existing BlackPearl customers. Existing BlackPearl customers should work with Spectra Logic Support to ensure that the upgrade goes smoothly.

We explained the main new features of BlackPearl in two recent blog posts:

To assist developers with building BlackPearl 3.0 clients, we have released the following new tools and resources:

  • 3.0 Simulator -- simulates a working BlackPearl for easy development
  • 3.0 API Reference -- review the operations supported by BlackPearl’s API
  • 3.0 Java SDK -- Use the new SDK to take advantage of all the new BlackPearl features
  • 3.0 Java SDK Javadocs -- Learn about all the SDK classes, methods, properties, and events

Over the coming weeks we will be releasing 3.0 versions of our SDKs in our remaining languages -- .NET/C#, C, and Python. Note that all of the existing 1.x SDKs work fine with BlackPearl 3.0, they just don’t yet have access to the new 3.0 features.

We will be releasing more resources and information over time to help you more easily build BlackPearl 3.0 clients.

 

BlackPearl Certification Program

Spectra Logic has created the BlackPearl Certification Program to ensure that the BlackPearl clients created by our partners are robust and reliable. We have provided detailed information on how to get certified. All partners should strive to have their BlackPearl clients certified. We are now certifying our first clients and will be announcing them soon. If you are interested in having your client certified, please Contact Us.

BlackPearl Certification Program

 

Developer Program Team at NAB 2016

The biggest trade show in the Media and Entertainment arena, NAB 2016, is coming next month to Las Vegas, and Spectra Logic will be there in full force. The Spectra Logic Developer Program team will also be on site to answer your questions and discuss your BlackPearl client integration plans. Included in our team will be Ryan Moore, BlackPearl SDK Engineering Manager, and Jeff Braunstein, BlackPearl Developer Evangelist. Come visit us at booth SL11816 and tell us about your plans to integrate with BlackPearl.

 

New Article About Exceptions and Unexpected Conditions

Any software application has to deal with errors, exceptions, and unexpected conditions. BlackPearl client developers must consider what unexpected conditions and exceptions may occur in their clients and have the client respond to them appropriately. We have created a new article that reviews the most common unexpected conditions in a BlackPearl client and how to respond to them appropriately. We recommend all developers read it and ensure that their clients follow these guidelines.

Managing Unexpected Conditions in BlackPearl Applications

 

BlackPearl Object Naming

As with most cloud storage systems, when a file is uploaded to BlackPearl we call it an “object”. There are some constraints around how objects are named in BlackPearl due to it typically being connected to a tape system that stores data using the open LTFS format.

When you configure BlackPearl, you set up Storage Domain(s), which are collections of tape and/or disk partitions. If the Storage Domain includes tape partition(s), you must specify the “LTFS File Name” option for the Storage Domain. This option specifies how the file is named when it is placed on tape. There are two options for the LTFS File Name:

  • Object Name — LTFS file names use the format {bucket name}/{object name}, for example bucket1/video1.mov. Object names must comply with LTFS specification file naming rules. If the tapes are ejected from the BlackPearl gateway and loaded into a non‐BlackPearl tape partition, the file names match the object names. The colon character (:) is not allowed in LTFS file names and therefore not allowed in BlackPearl object names. The slash character (/) is also technically not allowed in LTFS files names; however, in BlackPearl a slash is allowed in the object name and will get translated as a directory in the LTFS file system (e.g. directory1/directory2/video1.mov). The following characters are not recommended in LTFS file names or BlackPearl object names for reasons of cross-platform compatibility: control characters such as carriage return (CR) and line feed (LF), double quotation mark (“), asterisk (*), question mark (?), less than sign (<), greater than sign (>),  backslash (\), vertical line (|)
  • Object ID — LTFS file names use the format {bucket name}/{object id}, for example bucket1/1fc6f09c‐dd72‐41ea‐8043‐0491ab8a6d82. Object names do not need to comply with LTFS file naming rules. The object names are saved as LTFS extended attributes allowing any third party application to reconstruct all the data including the object names.

When building a BlackPearl client, developers should be prepared for customers to choose either LTFS File Name option, and thus should ensure that object names are LTFS compliant as described in the first option above.

 

Java CLI Updates – Symbolic Link Support, Performance Test

We have released an update to the Java CLI BlackPearl client. You can download the latest release from the Java CLI Releases page on GitHub. There are several updates in the new release, including two major new features.

Read More

 

BlackPearl “Chunks” and “Blobs”

BlackPearl uses the Spectra S3 protocol to manage files. We receive many questions on how files interact with BlackPearl using this protocol, and particularly how files are broken up for moving them to and from BlackPearl. Hopefully this post will help clarify the topic.

Read More

 

SDK & CLI New Releases; Improvements to BlackPearl 3.0 Compatibility

Our Engineering Team has released new versions of the .NET/C# and Java Software Development Kits (SDK), as well as the Java Command-Line Interface (CLI). There are a number of new features and bug fixes in these releases. Probably the most important new feature in these releases is improved support for the BlackPearl 3.0. We strongly recommend that developers currently building BlackPearl clients using the .NET or Java SDKs upgrade to these new SDKs so that their clients will be more compatible with BlackPearl 3.0. These new releases should not require any code changes.

Download Latest .NET SDK

Download Latest Java SDK

Download Latest Java CLI

 

BlackPearl Performance, File Size, and Job Size

BlackPearl uses Bulk PUT and GET commands to move files to and from its storage targets -- tape and disk. We call them “Bulk” commands because they transfer multiple files in one “job”. The bulk commands provide several advantages over traditional, single-file S3 PUT and GET operations, including: 1) ensuring that the BlackPearl cache is ready to receive the files; and 2) providing adequate data to continuously write or read data on to the storage target at a high level of performance.

Many factors affect the file transfer performance to and from BlackPearl, including network configuration and equipment specifications, transfer rate capabilities from the primary storage medium, file transfer software architecture and file sizes/job size. To maximize file transfer performance (typically measured in megabytes per second) on a bulk PUT or GET job, the size of both the individual files in the job and the total job must be considered. Spectra Logic has performed considerable testing and analysis of various files sizes and has determined ideal file sizes for maximizing performance in bulk PUT and GET jobs. The results of this testing are included in this post.

Read More


Send Us Your Updates, Questions
Have you completed a Spectra S3 client? Need help on client development? Let us know what you’re doing! Send an update or ask for help on our Contact Page. And don’t forget about our Google Discussion Forum to ask for help from our team and your peers.

 

 
     
Spectra Logic | 6285 Lookout Road | Boulder, CO 80301 | 1.800.833.1132 | SpectraLogic.com
View this email in a browser. Click here if you don't wish to receive these messages in the future.