RAW Test Report

Test plan

RAW applicationhttp://app.raw.densitydesign.org/#%2F

Testing will be performed against RAW – Latest Git Commit 3555470, Mac OSX Sierra 10.12 & Chrome 54.0.2840.59

Test charter

The aim of the test session is to get broad coverage of the high level features of the RAW application. The focus of testing will be on verifying basic functionality to confirm that features function according to expectations derived from the user documentation and product marketing.

Test approach

  • Exploratory tests
  • Functional tests against requirements inferred from the FAQs & API reference docs
  • Test notes will be recorded in spreadsheet, for conversion via RAW
  • Test data will be manually created to push the boundaries of the system (attached)
  • Test data will be sourced from sites like NASA and UN in order to collate real-world data to simulate real-world examples

In Scope

  • Application in single web page

Out of Scope

  • full RAW website (headers, footers, other pages)
  • Cross platform/browser
  • Custom developed reports
  • API
  • Automation

Note: Conditions under test are limited due to time constraints

High level tests

  • Parse data
    • Copy/paste data
    • Drag/drop data
    • Valid data
    • Invalid data
    • File formats
    • Cell formats
    • Check results in Text format
    • Check results Tabular format
    • Sample data from RAW
  • Create Visualisation
  • Configure Visualisation
  • Export Visualisation
  • Performance – large files
  • Security – is data stored on servers (monitor http requests)

Test Inputs

hand crafted data (originally a numbers document, exported to .xls)

screen-shot-2016-10-21-at-6-28-23-am

Test Results

Test cases

The RAW created image displays the test cases & results:

test_cases

Defects

During testing, I found the following issues to be of concern, and worthy of further investigation as potential bugs:

  • Download Visualisation: Unable to download file that contains foreign chars (Error: failed to execute ‘btoa’ on Window: The string to be encoded contains characters outside the literal range.) (tree).
  • Download Visualisation: When there is an error downloading report, user gets no front end error message – page doesn’t nothing (error only available in console)
  • Download Visualisation: If file name contains spaces, file is not downloaded successfully
  • Visualisation: When labels do not fit to cell, there is no text wrapping (tessellation)
  • Visualisation: Text does not fit to cell, even when setting width very wide (tree)
  • Visualisation: Text overlapping (unable to edit)
  • Configure Visualisation: No upper limit for Width, Height, Padding fields (can cause browser performance issues such as freezing)
  • Configure Visualisation: No validation on Width, Height, Padding fields (can cause report to be generated with invalid params)
  • If report contains invalid dimension mapping, Download link is enabled (nothing to download)

Sample Defect Report

001 – Tabular data format – text view sort order

Priority: Low – as the main purpose of the programme is to create visual representations of the data, the sort order on the table view format of the raw data may not be high priority. However, users with non tabular, delimited data may require table format sort order as per original.

Description: 

When user copies/drops data to the RAW editor, the user specified sort order is not retained in table view. An alphabetical sort order is always applied.

Steps to reproduce: 

Pre-requisites: source data contains data where rows / columns are not in alphabetical order

  1. Copy/Paste or Drop file to RAW editor
  2. Check the display order on the source file and the text view
  3. Check the display order on the table view

Expected: 

Table view displays data in same sort order as user’s original data. User can optionally apply sort order by row or column.

Usability feedback

  1.  Parsing large file – no “loading” message

    When user is copying large file to RAW, the application can “freeze” while processing, but user is not alerted with a processing message or image.

  2. Unable to delete a column/row of data from tabular view
  3. Visualisation: Unable to edit tree nodes padding / distance (tree)
  4. mapping invalid dimensions for a visualisation – error feedback

    if user maps an invalid dimension (as per screenshots below), user sees:

  • small grey panel with error message “Don’t drop here”
  • small grey font with error message “You should only use X or Y here”
  • invalid dimension is highlighted in yellow

screen-shot-2016-10-20-at-6-51-40-pmscreen-shot-2016-10-20-at-6-52-10-pm

Feedback: Because the grey font and yellow fields are not typical colours associated with errors, it could be difficult for some users to notice that an error has been created.

5.  visualisation not rendered with no error feedback
if user maps an invalid dimension, the visualisation cannot be rendered. In this case, user sees:

  • visualisation field displays blank with no error feedback

screen-shot-2016-10-20-at-8-48-17-pm

Feedback: The lack of error feedback regarding invalid dimension mapping in this section could cause confusion for users regarding why the visualisation hasn’t rendered correctly.