Frequently Asked Questions
These are questions that have cropped up before, and can hopefully solve your query
Noteable FAQs
Absolutely. The way you do it is slightly different for student & instructors
For a student, you can delete the folder <course code>/<assignment code>
For an instructor it's more complicated: There is data in your grading database, your original files, and the multiple student submissions & graded files.
The easy solution is to used the archive-assignment
script from a terminal session.
The situation is described in more details in the "Technical questions" section below
Noteable High-Level Design Description
First, we have to declare that the term 'notebook' is often over-used, so let's break this down into four parts:
- The 'notebook document' is the file, in
ipynb
format, that contains the source code & output of your work. - There's the backend bit that takes the source code and compiles it. This is language-specific, and is called the 'kernel'.
- There's the front-end 'notebook dashboard', which is mainly used to open notebook documents and manage files.
- Finally, the 'notebook app' is the term for the application that provides the web-server which powers all this.
Let's start with 3rd generation programming languages (3GL) - machine independent, abstracted, programmer-centric. The source code is written, then compiled into another form, which is then executed.
3GL programmers often use an integrated development environment (IDE) tool to write, compile, and execute code all from the same place. IDEs often come with additional helpful features such as syntax highlighting, code-completion, and debugging. 3GL programming means a developer can use an IDE on their workstation, with locally installed libraries, to create a compiled application that can be distributed & executed as is on other computers.
iPython took the renowned programming language Python and wrapped an interactive shell around it - so it would compile & execute sections of code in sections. iPython came with a command-line shell, and a game-changing IDE: a web-based interface.
iPython's main goal is not application development, but data analysis, exploration & modelling.
Jupyter Notebooks are a language-agnostic IDE for iPython, and also bring a wealth of visual & user-centric improvements to the tool. These developments, primarily open-source and developer-community oriented, are the building blocks of the Noteable service.
A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries and settings.
The important part for us is 'includes everything needed to run an application'.
The problem with a Notebook App is that is runs on the user's computer, and needs the user to have all the libraries installed that they want to use in their notebook documents.
If we can put the Notebook App plus all the desired libraries into a docker container, we have a distributable service that we know is complete & correct.
Noteable is, at this level, two things:
- A mechanism to provide authenticated access to a range of different Docker App containers
- It uses a Jupyterhub that can launch notebook apps into a private, secure cloud
- Noteable provides the tools and support, for lecturers, students and researchers, whatever their level of skill or knowledge, to deliver narratively rich, coded solutions.
- Noteable enables computational narratives to be readily created, shared and reused within and across teaching and learning contexts.
- In order for data, and the computations that process and visualize that data to be useful and meaningful for humans, they must be embedded into a narrative - a computational narrative - that tells a story for a particular audience and context.
- Our tools are designed by lecturers and researchers to support lecturers and researchers.
- They are easily integrated into your existing Virtual Learning Environment solutions.
- Tailorable to your teaching: Notebooks can be discipline specific or designed to be bespoke for your needs.
- We can support researchers working with data as well: To provide more CPU or more data storage, niche language kernel support and Big Data tooling.
- Your student numbers may change but our infrastructure makes sure your Notebooks still work smoothly. All without recourse to local IT support.
We have a Trial Access system for potential customers.
When setting up Trial Access, you will be limited to the number of concurrent users (normally 10). Authentication is via organisation email address, and runs for an agreed time (normally a month).
Note: Trial access is via a your VLE/LMS, allowing you to control, and test, roles and privilages. Trial accounts are not carried over to a main subscription.
Technical questions
There is a technology called Learning Tool Interoperability which allows an organization's Virtual Learning Environment to connect an authenticated user to Noteable.
Setup process
This process is handled by the admin team for your VLE/LMS, and only needs to be done once
We will first set up a connection to the noteable-beta site, to confirm you can create the connection you want, and to allow your service-team to preview updates.
For an LTI connection, you will be setting up and External Tool
or basic LTI Tool
. We will sent you three pieces of information: a Key
, a secret
, and the Launch url
... and you plug these into the appropriate fields in your configuration form. You need to open in a new window [or tab] - and not stay in an iframe.
We look for four pieces of information:
- username
- context id (the course code)
- context title (the course name)
- role ('Instructor' or anything else)
We do not look for real names, email addresses or any other information.
We would prefer a username such as s2012345
to 35f17232b17b6a4d0025c64b53658088a1545860
, and a sensible course code (perhaps MATH101_2020_SEM1
) rather than 86fde15399c4156c8d0aee664b625d3de2920784
- but Noteable will work with the obfuscated codes.
Once the noteable-beta connection is working correctly, we will create a new Key
, secret
, and Launch url
to enable access to the main noteable service.
As a tutor/instructor - for classes
Your local VLE/LMS team are best placed to help with this, but essentially you will be adding an External Tool
or Basic LTI Tool
item to the set of modules & items in the course-content.
This assumes the libraries listed in the synopsis descriptions is not enough
Maintaining a complete & comprehensive table of libraries across all notebooks proved to be an enormous, and unwieldy, table - so, no, that list is not available outside the service.
However
If you start the relevant notebook, you can get the list yourself:
- run
!conda list
in a notebook cell, or - run
conda list
in a terminal
Likewise, installed.packages()
will list R libraries.
For most notebooks
In an early cell in your notebook, install the package - use either of these options:
!conda install -y package-name
!pip install package-name
conda
is much slower, but checks version dependencies to ensure there are no problems. pip
will just install the library.
In either case - the library will not be present when you next start a notebook server.
For RStudio
You can install Python packages as above (and many r-libraries are available as conda or pip libraries).
For libraries from the R repository, we have found the solution that works is to use the terminal
:
R -e 'install.packages('testthat', repo='http://cran.rstudio.com/')'
will install thetestthat
libraryR -e 'install.packages(c('testthat', 'alr3', 'funkyPigeon'), repo='http://cran.rstudio.com/')'
will install all three libraries
Packages are installed under $HOME/R/
and will persist between sessions
From your notebook, you can't.
Noteable notebooks have an extension called “nbgitpuller
” (available under the “+GitRepo
” button in the Notebook Dashboard.) If you make files available from an accessible git repo (does not have to be GitHub), then others may download those files. For full details on using this method, please read the nbgitpuller documentation. IMPORTANT: You do NOT need to create fancy links or anything... +GitRepo
does that for you.
Most (but not all) Noteable notebooks have an extension called “nbgrader
”. This allows instructors to make assignments available for courses. Nbgrader has the advantage that it supports grading and limits distribution to just those on a specific course. The downside is that nbgrader requires a connecton from an LMS/VLE (see the nbgrader section)
Noteable has different Notebooks available to the user.
The majority of notebooks have Python 3
with a number of additional libraries pre-installed.
Specialist notebooks provide specialist kernels: specifically R
Specialist notebooks provide different Python libraries: Geospatial
, R with Stan
, R in the RStudio
interface, ....
Short answer: no
Long answer: you can sort of fudge it.
The upstream nbgrader
solution is to have a shared notebook-server, which is not an available solution within Noteable.
The way to fudge it is to slice the problem into smaller segments:
- Vertical slicing: In this scheme, each tutor marks all questions for a subset of the class. Ensure your tutors are identified as
Instructors
when they access Noteable (ie, they get theformgrader
tab). You divide your student cohort across the tutors, and each tutor generates & releases their own assignment [assumed to be the same.ipynb
file, suitably named to distinguish it from the other tutors]. Students then only fetch the assignment for their group, and the process is effectively managed. Results will need to me manually amalgamated. - Horizontal slicing: This simply changes the marking process from all questions for a smaller group to all the students for a smaller assessment. As before, ensure your tutors are identified as Instructors - then each tutor releases a different notebook [just containing the code for their questions.] The downside to this solution is that questions need to be completely self-contained - and that's hard to do.
Using the archive-assignment
script:
The archive-assignment
script archives the assignment, and deletes it from your records. It will:
- Manage the database:
- copy the grades for an assignment into a
.csv
file - delete the assignment-related data from your local
gradebook.db
- copy the grades for an assignment into a
- Manage the files:
- create a
.zip
file of all the source, submission, grading, and feedback files that relate to this assignment - delete them from the disk.
- create a
How to:
- Start a new terminal (under the
New
menu) - Run
archive-assignment
to see the options for the command
If we assume the course code is made up
, and assignment is Test 1
, the command would be archive-assignment "Test 1"
This will create two files: made up.Test 1.csv
and made up.Test 1.zip
Manually remove all records
The process allows a more controlled, and informed, archiving process. All work is done in a terminal (under the New
menu)
(These examples assume the course code is made up
, and assignment is Test 1
)
Managing the database
Note: The database does not use the course code to distinguish assignments (which is why you can't repeat assignment names across courses.)
- To list the assignments you [your local database] knows about:
nbgrader db assignment list
- To extract the scores from the database for a given assignment (assume assignment name =
Test 1
, and we want to put the contents in a file calledmade up.Test 1.csv
):nbgrader export --assignment '["Test 1"]' --to "made up.Test 1.csv"
- To clear the data:
nbgrader db assignment remove "Test 1"
- (It will complain if the assignment has scores/grades recorded)
nbgrader db assignment remove "Test 1" --force
Managing files
Note: Files use the course code in some file paths
- Check what files you have:
ls source/"Test 1" release/"Test 1" autograded/*/"Test 1" made\ up/collected/*/"Test 1" feedback/*/"Test 1"
- If that looks reasonable, consider making a zip file of all the files:
zip -r "made up.Test 1.zip" source/"Test 1" release/"Test 1" autograded/*/"Test 1" "made up"/collected/*/"Test 1" feedback/*/"Test 1"
- Delete all the files (warning there is no recovery from this):
rm -rf source/"Test 1" release/"Test 1" autograded/*/"Test 1" "made up"/collected/*/"Test 1" feedback/*/"Test 1"