TOMUSS Technical Reference

Table of content:

TOMUSS file organization

The over top file organization

Running server file organisation

TOMUSS plugins

URL template

Plugin attributes

Plugin documentation

Plugin replacement

TOMUSS protocols

TOMUSS client/server protocol

TOMUSS suivi client/server protocol

TOMUSS storage

Table storage

ABJ + DA storage

TT storage

Referent storage

TOMUSS objects server side

C

Cell

CellEmpty

CellValue

CellVirtual

Column

Columns

Line

Lines

UE

Types of columns

Attributes of table columns

Attributes of tables

TOMUSS server side threads

TOMUSS client side

The table editing page

Client side objects

Virtual tables

The 'Display' framework

TOMUSS Translation

TOMUSS Administration

Semesters

Table templates

Authentication process

Apache/NGINX configuration

TOMUSS configuration

TOMUSS authorizations

TOMUSS Regressions Tests

TOMUSS starting and stopping

TOMUSS install on production server

TOMUSS managing

TOMUSS Pitfall

How to customize the display

Functions to redefine in order to customize TOMUSS

Some glossary :

TOMUSS file organization

The first part describes the hierarchy created by the default installation script and the second part describes the hierarchy of the running servers.

You may skip the first part is you only want to test TOMUSS

The over top file organization

The hierarchy created by the installation tree insure that a running server can be stopped and an older version of the server can be quickly started without data loss.

Running server file organisation

If the installation script is used, the following directories are symbolic links to the version independent directories.

The top level directories

The core of the code

TOMUSS plugins

When a TOMUSS process is started it loads its plugins. The received requests are dispatched to the fittest plugin. The most important attributes of a plugin are.

Do not modify plugins.py source to add plugins, use a redefine to add plugins

Do not modify global state with plugins because it may cause problems if the plugin is reloaded while the server is running.

URL template

It defines the pattern of the request. For example: {Y}/{S}/{U}/resume

Here are the components of the URLs and where they are stored in the request object in the server side.

ComponentAttribute nameComment
{Y}the_yearThe year of interest
{S}the_semesterThe semester of interest
{U}the_ueThe course of interest
{P}the_pageThe page of the table
{?}somethingNot important
{I}the_studentA student identifier
{_I}the_studentA student identifier prefixed by _
{*}the_pathThe remaining of the path
{=} Remove any /=.... from the path
{ }the_timeThe navigator request time (avoid caching)

Plugin attributes

Plugin usage can be restricted to a list of user groups using config_plugins table, user groups are defined with the config_acls table. The default authorized group list contains the value of the group parameter.

The possible values of the decision attributes are None to indicate that it is not important, True to indicate that session must match the attribute, False to indicate that session must not match the attribute.

The action attributes are:

Plugin documentation

Some of the plugins are used both in TOMUSS and 'suivi' servers.

The ``link and position´´ defines the data for the function generating the HTML content to insert in an home page. The position specify the name of the box where to insert the code.

Name and locationLink and positionExplanations
abj
abj_change.py:76
JLV and DA edition in abj_master
Go to the page allowing to edit the justified leaves and the non-attendance authorization.
Display the home page for ABJ management
abjaction
abj_change.py:130
This plugin perform an action on the ABJ and DA list. These actions can be adding or removing an ABJ or a DA. The client receive a feedback image indicating the status of the change.
abjalphaauthor
abj_change.py:112
JLV+DA you entered in abj_master
Send a CSV file containing all the ABJ and DA information for all the students
abjhacker
abj_change.py:163
Display an error message becauses this action is unauthorized
abjhtml
abj_change.py:89
Full JLV+DA report of students in abj_master
You must indicate a regular expression at the URL end defining the UE code that need to be listed. A single matching UE will make the student listed.
Send an HTML file containing all the ABJ and DA information for the students registered to the specified UE regular expression. Add to 'config_home' table some link : * with container 'abj_master'; * the 'verysafe' class; * the group name needing to see the link 'abj_masters' For example: Master: Full JLV+DA report javascript:go('abj/html/.*M$24') Licence: Full JLV+DA report javascript:go('abj/html/.*L$24') EPU: Full JLV+DA report javascript:go('abj/html/UE-EI') $24 because the dollar must be escaped.
abjlistmail
abj_change.py:123
Messages for the masters of UE licence in abj_master
Allow to send a message to all the masters of UE of licence. The message contains a report about justified leaves, non-attendance authorization and special examination condition, linked to the UE.
Generate and display all the message to the UE teachers given them the names of the students with ABJ, DA and TT.

At the end of the page, there is a link to really send the messages

abjsendmail
abj_change.py:118
When triggered this function send all the mail generated by the plugin listmail.
accept
suivi_student.py:539
The student signs the contract
acls_check
answer_page
newpage.py:153
Connect the browser IFRAME to the page
atomic_checker
gc_state.py:349
Size of all str, int... in debug
Compute the memory size of untracked by the GC atomic objects
Compute the size of atomic Python type : integer, string
auto_update
auto_update.py:29
Update students in tables in root_rw
Update the student list for each table of the current semester.
Update the student list for each known table of the current semester. Or the tables in the path in the current semester.
auto_update_one
auto_update.py:29
Update the student list for each known table of the current semester. Or the tables in the path in the current semester.
backtrace
backtrace.py:189
Display backtraces in debug
Navigate in the error reports
Display backtraces. Arguments are: {list|day}/{any|important|warning|informative}/fromYYYYMMDD/until
bad_ip
suivi_bad_students.py:67
Student not officially registered in abj_master
Display the list of students without official registration, but with grades entered in tables.
Display the list of student attending courses but not officialy registered
badname
suivi_badname.py:31
Bad names in deprecated
Show the names not matching the student ID in the tables
Display the names not matching the student ID.
badpassword
badpassword.py:25
Display an information page to indicate that the user password is not safe because it is too simple. So the user has not access to the software
bilan
bilan.py:35
Displays all the informations about a student in TOMUSS for all the semesters. These information can be augmented with other data
bookmark
pageaction.py:159
Toggle the bookmarked state of a table
caches
gc_state.py:173
Display the caches in debug
Useful to find memory leaks
Display the caches statistics
cell
cell_change.py:56
Modify an unique cell in a table. It is used by the 'suivi' page
change_identity
change_identity.py:26
Take the identity of:<form style="margin:0" action="javascript:var m = document.getElementById('new_identity').value ; window.location=add_ticket('change_identity/' + m)"><input id="new_identity" class="search_field" name="x" class="keyword" value="john.doe"></form> in debug
Really take the identity of another user
Change of identity for this ticket
check_ticket
clean
clean.py:25
Delete empty UE in root_rw
Deleted UE are moved in Trash directory. This operation because it is slow and It is done only in the current chosen year/semester.
Remove all the UE that where not modified and are not actives. Display the list of the tables and why they are not deleted. Tables with lines are not deleted even if empty.
clean_other
clean.py:53
Delete empty tables in root_rw
Deleted tables are moved in Trash directory. Avoid this operation because it is slow. This operation does not delete UE.
Remove all the tables (not UE) that are completly empty.
clients
clients.py:121
Browser statistics in informations
About browser, OS, IP addresses. Really not fine for privacy
Display client statistics.
column_speak
comment
count
count.py:35
count the number of cells with the given value used in the given tables per weeks. The week date must be indicated in the column course date. It may be used to compute the number of ABINJ per week for a set of UE. http://127.0.0.1:SUIVI_PORT_NUMBER/count/ABINJ/UE-INF1001L/UE-INF1002L
create_table
create_table.py:34
Create a table in root_rw
You can invite students or teachers to participate.
Return the column definition of the table
delete_this_table
tabledelete.py:32
Delete the table.
emptyname
Bad url, redirect user to the home page
end_of_load
pageaction.py:196
Keep track of the end of page loading to do stats. If the page takes too long to load, then 'newpage.py' assumes that something blocked TOMUSS because the page loading does not finish.
evaluate
evaluate.py:25
Evaluate a Python statement and display result
extension
pageaction.py:105
Extend the current table to the next semester. A symbolic link is created and the current table is accessible in the 2 semesters.
favorite_student
favorite_student.py:27
Toggle the favorite state of a student
fix_modifiable
fix_modifiables
fix_modifiables.py:31
Modifiable tables that should not be in root_rw
Fill the column with 'NO' to put the tables in read only
List of the modifiable OLD table
fusion
suivi_extract.py:192
Fusion of named columns from tables, display as an HTML table /fusion/UE-INF2011L:a/UE-INF2012L:
gcbig
gc_state.py:234
Display big Python object in memory. Use len() to find them. in debug
Useful to find memory leaks
Display big Python objects
gcobject
gc_state.py:119
Display the graph of instances using or used by the object
gctop
gc_state.py:35
Display Python object in memory in debug
Useful to find memory leaks
Display a clickable list of Python classes and their number of instance.
gctype
gc_state.py:73
Display a clickable list of the instance of the classes specified
get_columns
get_columns.py:27
Return the column definition of the table
groupe
suivi_groupe.py:53
All the groups defined by teachers in grouping
Get the group affectation done by teacher in all the tables
List all the students groups defined by the teacher and not by TOMUSS
gui_record
log.py:54
Store a text in a log file
home
suivi_student.py:567
Display the home page for 'suivi', it asks the student id.
homepage3
icone
import_zip
columnimportzip.py:117
Upload many files at once.
infos
suivi_student.py:579
Display the informations about all the students indicated.
invitation
invitation_accept
ip
suivi_ip.py:26
Deduce registration in deprecated
Find students registration from table content
For each student, display the tables where he's present
key_history
pageaction.py:182
Store the received data in the key history. It is used in 'linear' interface to analyse how it is used.
live_log
live_log.py:25
Live log in debug
Real time display of logs
This page continuously display logs of the server.
live_status_svg
live_status.py:30
Graphic state in debug
Real time graphic display of logs
This page continuously display logs of the server.
locks
gc_state.py:211
Display locks in debug
Useful to find what is locked
Displays all the lock states
log
log.py:30
Store a text in a log file
login_list
login_list.py:57
Return the list of teacher and students matching the search query
logout
logout.py:25
User deconnexion from the server.
master_of
master_of.py:29
Recompute masters index in root_rw
To use only in case of. It will compute for each user the list of table for which he is a master.
Update the preferences to add the list of UE where the user is master. This function is only used in case of bug in the algorithm updating the master list incrementaly
memory_size_checker
gc_state.py:314
Search memory leaks in debug
Each time the memory usage increase, display thread stacks
Display thread list each time the memory footprint change
my_picture
picture.py:124
Display the connected user picture or a teacher picture
my_picture_icon
my_picture_upload
Change your ID picture in root_rw
Your picture will be displayed to student on their suivi page.
number
count.py:143
count the number of cells with the given value used in the given tables. It may be used to compute the number of ABINJ for a set of UE. http://127.0.0.1:SUIVI_PORT_NUMBER/number/ABINJ/UE-INF1001L/UE-INF1002L
one_groupe
suivi_groupe.py:120
Group affectation for one table
orphan_students
referent_get.py:127
Remove the referent teacher of a set of student in referents
Remove the teacher of students
page_unload
pageaction.py:91
Unload page from memory
pageaction
pageaction.py:40
An editing action on the table is queued. The client may not send the action in the good order.
pagenew
newpage.py:52
Create a new page and send the table editor to the client.
pageresume
pageaction.py:66
Display the list of ABJ, DA and TT for the students in the table. It is the same list being sended by mail.
pagerewrite
pageaction.py:32
Generate the minimal python module defining the current table state. So there is no more history.
picture
picture.py:91
Display the picture of a student
picture_icon
preferences
suivi_preferences.py:42
User preference merging in informations
To see how users modify their preferences
Join of all the preferences table
profiling
utilities.py:92
Plugins profiling in debug
Compute the plugin response time
Display the statistics on the plugin usage, number of call and times.
public
public_login
referent
suivi_referent.py:28
Display the referent of the named student
referent_get
referent_get.py:78
Add a student to its refered students
referent_set
referent_get.py:95
Give a referent to a student set in referents
Students must not yet have a referent
Add a student to a referent : ...../referent_set/user.name/student1/student2/...
referent_set_force
referents
suivi_referents.py:147
Referent teacher statistics in referents
display statistics about referent teachers
Display statistics about referents.
referents.csv
suivi_referent_list.py:73
Generate the referent list in CSV
referents.html
suivi_referent_list.py:81
Refered student list in HTML in referents
Table giving for each student the informations about their referent
Generate the referent list in HTML
referents.xls
suivi_referent_list.py:81
Refered student list in XLS in referents
Table giving for each student the informations about their referent
Generate the referent list in HTML
referents_update
referent_update.py:25
Find referent teacher for students in referents
It searches a referent teacher for students without one
Dispatch students without referents to a referent.
referents_update_R
referent_update.py:38
Remove the referent teacher of old students in referents
This does not link referent to new students.
Dispatch students without referents to a referent.
referents_update_safe
referent_update.py:51
Displays the new affectation a student/referent in referents
No changes are done.
Dispatch students without referents to a referent.
referents_update_safe_R
referent_update.py:64
Displays the students that will be removed to referents. in referents
Dispatch students without referents to a referent.
reload_plugins
reload_plugins.py:34
Reload the plugins in debug
Reload all the modified Python code on disk in the directories: PLUGINS, ATTRIBUTES, COLUMN_TYPES and all the JavaScript and CSS. Module launching threads when loaded must not be reloaded...
Reload all the plugins from PLUGINS, COLUMN_TYPES, ATTRIBUTES
remove_history
tablecopy.py:110
Remove the history of the table. Do not call from a thread. Beware: users with non unclosed table will have bad_allow.html answer when trying to use the old table with the new one.
requests
gc_state.py:274
Display pending requests in debug
This allow to find values that can be lost because the web browser has not sent the requests in the good order before being closed.
Displays the requests queue
restart_tomuss
restart.py:35
Restart TOMUSS in debug
Wait to be in an unused state before restarting or stopping TOMUSS
Restart or stop TOMUSS when it is unused
resume
resume.py:35
Resume the number of cells used in the given tables
rss
suivi_student.py:658
RSS for the student.
rss2
suivi_student.py:790
RSS for the table.
rsskey
rsskey.py:31
Create and return the RSS key of the connected user
save_preferences
save_preferences.py:34
Set the user preferences
search
Opensearch results
send_alert
send_alert.py:27
Send the following message:<br><form style="margin:0" action="javascript:var m = document.getElementById('message').value ; if(confirm(_('LINK_send_alert_before') + '\n\n' + m)) window.location=add_ticket('send_alert//' + m)"><input id="message" class="search_field" name="x" class="keyword" value="The TOMUSS server will be restarted in a couple a seconds. If it is possible, reload the page in a minute to be on the safe side."></form> in root_rw
You can edit the text to be sent in an alert box to the users of the table editor. Hit 'Enter' to send the message.
Send an alert popup to the TABLE clients or all clients .
send_mail
send_mail.py:52
Send personnalized mails
signature
signature.py:231
Store the fact that the student has signed the question
signature_new
signature.py:87
Add a question to ask to the student
signatures
signature.py:248
Display all the signatures done by the student
statpage
statpage.py:38
Who's doing what? in informations
List tables currently used
Display the tables in memory and the user on them.
stop_tomuss
Stop TOMUSS in debug
Wait to be in an unused state before stopping TOMUSS
student
suivi_student.py:526
Display all the informations about a student.
student_redirect
Redirect the student on the 'suivi'.
suivi_extract
suivi_extract.py:101
Extract named columns from tables, display as an HTML table /extract/UE-YYYYYYY:Column1:Column2/UE-ZZZZZZZ:ColumnX:ColumnY...
suivi_fusion_inscrit_author
suivi_extract.py:203
Fusion of named columns from tables, display as an HTML table /fusion_inscrit_author/UE-INF2011L:a/UE-INF2012L:
tablebuffer
statpage.py:99
Display the tables in memory and the user on them.
tablecopy
tablecopy.py:42
Copy the table in another EMPTY one
tables
suivi_tables.py:30
Table Statistics in informations
Display statistics about each table.
Create a table of statistics about all the tables.
teachers
suivi_teachers.py:31
Teacher statistics in informations
For each teacher, display statistics about TOMUSS usage.
Create a table of statistics about all the teachers
threads
gc_state.py:223
Display the threads in debug
Useful to find is loading the CPU
Displays the running thread
tickets
tickets.py:29
Active tickets in informations
List the tickets currently active
Display tickets
timeline
tmp
uninterested
suivi_uninterested.py:32
Refered students statistics about TOMUSS usage in informations
For each students with a referred teacher, count the number of TOMUSS visit and grades.
Display information about student and their referents and how student interact with TOMUSS and referents.
unload
unload.py:25
Unload the table from memory.
upload_get
upload_get_public
upload_pdf
upload_post
upload_zip

Plugin replacement

If you want to replace a builtin TOMUSS plugin by yours, you must remove the standard one before defining your replacement. To delete the 'picture' plugin:

plugin.plugins.remove(plugin.get("picture"))

TOMUSS protocols

The client perform action on the server by inserting image objects in the HTML code. The returned image give the user feedback (green: ok, orange: waiting server, red: unauthorized, violet: bug...)

The client does not poll server to retrieve changes, but the server send javascript fragment when there is an update to send.

TOMUSS client/server protocol

The client load the documents as usual. But if the document is a table:

NameURL templateAuthenticated Group allowed Password OK Backgrounded Cached Keep open Mime Type
abjaction{Y}/{S}/abj/{P}/{I}/{*}Tabj_mastersTFFFtext/html; charset=UTF-8
abjhacker{Y}/{S}/abjsTTFFFtext/html; charset=UTF-8
save_preferencessave_preferences/{*}TTFFFimage/png
check_ticketcheck_ticket/{?}FTFFFapplication/x-javascript; charset=UTF-8
invitation_acceptinvitation_accept/{*}TTFFFtext/html; charset=UTF-8
login_listlogin_list/{*}TstaffTTFFapplication/x-javascript; charset=UTF-8
my_picture_uploadmy_picture_uploadTTTFFtext/html; charset=UTF-8
picturepicture/{?}TstaffTTTFimage/jpeg
my_picturepicture/{?}T!staffTTTFimage/jpeg
my_picture_iconpicture-icon/{?}T!staffTTTFimage/jpeg
rsskeyrsskeyT FFFtext/plain; charset=utf-8
signaturesignature/{Y}/{?}T!staffTFFFimage/png
cell{Y}/{S}/{U}/cell/{*}TTFFFtext/html; charset=UTF-8
comment{Y}/{S}/{U}/comment/{*}TTFFFtext/html; charset=UTF-8
upload_get{Y}/{S}/{U}/upload_get/{*}TTTFFNone
upload_get_public{Y}/{S}/{U}/upload_get_public/{*}FTTFFNone
upload_post{Y}/{S}/{U}/upload_post/{*}TTTFFtext/html; charset=UTF-8
student_redirect{*}T!staff FFF307
abj{Y}/{S}/abjTabj_mastersTFFFtext/html; charset=UTF-8
abjalphaauthor{Y}/{S}/abj/alpha_author.xlsTabj_mastersTTFFtext/html; charset=UTF-8
abjhtml{Y}/{S}/abj/html/{*}Tabj_mastersTTFFtext/html; charset=UTF-8
abjlistmail{Y}/{S}/abj/list_mailTabj_mastersTTFFtext/html; charset=UTF-8
abjsendmail{Y}/{S}/abj/send_mailTabj_mastersTTFFtext/html; charset=UTF-8
end_of_load{Y}/{S}/{U}/{P}/end_of_loadTstaffTFFFimage/png
key_history{Y}/{S}/{U}/{P}/key_history/{*}TstaffTFFFNone
answer_page{Y}/{S}/{U}/{P}/{R}TstaffTFFTtext/html; charset=UTF-8
pageaction{Y}/{S}/{U}/{P}/{*}TstaffTFTTimage/png
acls_checkacls_checkTrootsTTFFtext/html; charset=UTF-8
atomic_checkeratomic_checkerTrootsTTFFtext/html; charset=UTF-8
auto_updateauto_updateTrootsTTFFtext/html; charset=UTF-8
auto_update_oneauto_update/{*}TstaffTTFFtext/html; charset=UTF-8
backtracebacktrace/{*}TrootsTTFFtext/html; charset=UTF-8
bilanbilan/{I}TreferentsTFFFtext/html; charset=UTF-8
cachescachesTrootsTFFFtext/html; charset=UTF-8
change_identitychange_identity/{*}TrootsTTFFtext/html; charset=UTF-8
clean_otherclean_otherTrootsTTFFtext/html; charset=UTF-8
clientsclients/{Y}TrootsTTFFtext/html; charset=UTF-8
create_tablecreate_table/{*}TstaffTFFFtext/html; charset=UTF-8
evaluateevaluate/{*}TrootsTFFFtext/plain; charset=utf-8
favorite_studentfavorite_student/{?}TstaffTFFFimage/png
fix_modifiablefix_modifiable/{Y}/{S}/{U}TrootsTFFFimage/png
fix_modifiablesfix_modifiablesTrootsTTFFtext/html; charset=UTF-8
gctopgcTrootsTTFFtext/html; charset=UTF-8
gcbiggcbigTrootsTTFFtext/html; charset=UTF-8
gui_recordgui_recordTstaffTTFFtext/html; charset=UTF-8
live_loglive_logTrootsTTFTtext/plain;charset=UTF-8
live_status_svglive_status.svgTrootsTFFTimage/svg+xml;charset=UTF-8
lockslocksTrootsTFFFtext/html; charset=UTF-8
loglog/{*}FstaffTFFFtext/html; charset=UTF-8
logoutlogoutT FFF307
master_ofmaster_ofTrootsTTFFtext/html; charset=UTF-8
memory_size_checkermemory_size_checkerTrootsTTFFtext/html; charset=UTF-8
gcobjectobject/{*}TrootsTTFFimage/png
orphan_studentsorphan_students/{*}Treferent_mastersTFFFtext/plain; charset=UTF-8
picture_iconpicture-icon/{?}TstaffTTTFimage/jpeg
profilingprofiling/{Y}TrootsTTFFtext/html; charset=UTF-8
referentreferent/{?}TstaffTFFFtext/html; charset=UTF-8
referent_getreferent_get/{*}TreferentsTFFFtext/plain; charset=UTF-8
referent_setreferent_set/{*}Treferent_mastersTFFFtext/plain; charset=UTF-8
referent_set_forcereferent_set_force/{*}Treferent_mastersTFFFtext/plain; charset=UTF-8
referents_updatereferentsTreferent_mastersTTFFtext/html; charset=UTF-8
referents_update_Rreferents_RTreferent_mastersTTFFtext/html; charset=UTF-8
referents_update_safereferents_safeTreferent_mastersTTFFtext/html; charset=UTF-8
referents_update_safe_Rreferents_safe_RTreferent_mastersTTFFtext/html; charset=UTF-8
reload_pluginsreload_pluginsTrootsTFFFtext/html; charset=UTF-8
requestsrequestsTrootsTFFFtext/html; charset=UTF-8
restart_tomussrestart_tomussTrootsTTFFtext/html; charset=UTF-8
searchsearch/{*}TTTFFtext/html; charset=UTF-8
send_alertsend_alert/{U}/{*}TrootsTFFFtext/html; charset=UTF-8
send_mailsend_mailTstaffTTFFtext/html; charset=UTF-8
signature_newsignature_new/{I}/{ }/{?}TstaffTFFFtext/html; charset=UTF-8
signaturessignatures/{I}TTFFFtext/html; charset=UTF-8
statpagestatTrootsTFFFtext/html; charset=UTF-8
stop_tomussstop_tomussTrootsTTFFtext/html; charset=UTF-8
tablebuffertablebuffer/{Y}/{S}/{U}TrootsTFFFtext/html; charset=UTF-8
threadsthreadsTrootsTFFFtext/html; charset=UTF-8
ticketsticketsTrootsTTFFtext/html; charset=UTF-8
tmptmp/{*}TTFFFimage/png
gctypetype/{*}TrootsTTFFtext/html; charset=UTF-8
badpassword{*}TstaffFFFFtext/html; charset=UTF-8
homepage3{=}TstaffTTFFtext/html; charset=UTF-8
emptyname{Y}/{S}/TTFFF307
clean{Y}/{S}/cleanTrootsTTFFtext/html; charset=UTF-8
bookmark{Y}/{S}/{U}/bookmarkTstaffTFFFimage/png
column_speak{Y}/{S}/{U}/column_speak/{*}TstaffTTFFaudio/x-wav
delete_this_table{Y}/{S}/{U}/delete_this_tableTstaffTFFFtext/html; charset=UTF-8
extension{Y}/{S}/{U}/extensionTstaffTFFFtext/html; charset=UTF-8
get_columns{Y}/{S}/{U}/get_columnsTstaffTFFFapplication/x-javascript; charset=UTF-8
import_zip{Y}/{S}/{U}/import_zip/{*}TstaffTTFFtext/html; charset=UTF-8
invitation{Y}/{S}/{U}/invitationTstaffTTFFtext/html; charset=UTF-8
page_unload{Y}/{S}/{U}/page_unloadTrootsTFFFtext/html; charset=UTF-8
remove_history{Y}/{S}/{U}/remove_history/{*}TstaffTFFFtext/html; charset=UTF-8
pageresume{Y}/{S}/{U}/resumeTstaffTTFFtext/plain; charset=UTF-8
pagerewrite{Y}/{S}/{U}/rewriteTstaffTTFFtext/plain; charset=UTF-8
tablecopy{Y}/{S}/{U}/tablecopy/{*}TstaffTFFFtext/html; charset=UTF-8
upload_pdf{Y}/{S}/{U}/upload_pdf/{P}/{*}TstaffTTFFtext/html; charset=UTF-8
upload_zip{Y}/{S}/{U}/upload_zip/{*}TstaffTTFFapplication/zip
pagenew{Y}/{S}/{U}/{=}TstaffTTFTtext/html; charset=UTF-8

TOMUSS suivi client/server protocol

NameURL templateAuthenticated Group allowed Password OK Backgrounded Cached Keep open Mime Type
icone{_I}TstaffTFTFimage/png
homeTstaffTFFFtext/html; charset=UTF-8
teachers*TstaffTTFFtext/html; charset=UTF-8
bad_ip*1Tabj_mastersTTFFtext/html; charset=UTF-8
tables*2TstaffTTFFtext/html; charset=UTF-8
referents*3TrootsTTFFtext/html; charset=UTF-8
acceptacceptT!staff TFFtext/html; charset=UTF-8
badnamebadnameTrootsTTFFtext/html; charset=UTF-8
countcount/{*}TstaffTTFFtext/html; charset=UTF-8
suivi_extractextract/{*}TstaffTTFFtext/html; charset=UTF-8
fusionfusion/{*}TstaffTTFFtext/html; charset=UTF-8
suivi_fusion_inscrit_authorfusion_inscrit_author/{*}TstaffTTFFtext/html; charset=UTF-8
groupegroupeTabj_mastersTTFFtext/html; charset=UTF-8
one_groupegroupe/{*}TstaffTTFFtext/csv; charset=utf-8
ipipTrootsTTFFtext/plain; charset=utf-8
numbernumber/{*}TstaffTTFFtext/html; charset=UTF-8
publicpublic/{Y}/{S}/{U}FTTFFtext/html; charset=UTF-8
public_loginpublic_login/{Y}/{S}/{U}TTTFFtext/html; charset=UTF-8
referents.csvreferents.csvFstaffTTFFtext/csv; charset=utf-8
referents.htmlreferents.htmlFstaffTTFFtext/html; charset=utf-8
referents.xlsreferents.xlsFstaffTTFFapplication/excel; charset=utf-8
resumeresume/{*}TstaffTTFFtext/html; charset=UTF-8
rssrss/{*}F FFFNone
rss2rss2/{*}F FFFapplication/rss+xml; charset=utf-8
preferencesstat_preferencesTrootsTTFFtext/html; charset=UTF-8
timelinetimeline/{I}TstaffTFFFtext/html; charset=UTF-8
uninteresteduninterestedTrootsTTFFtext/html; charset=UTF-8
unloadunload/{U}TTFFFtext/html; charset=UTF-8
student{*}T!staff TFFtext/html; charset=UTF-8
infos{*}Tstaff TFFtext/html; charset=UTF-8

TOMUSS storage

The table data is stored as a Python module, the module source is only modified by doing an append to prevent any data loss. File size is checked before and after append to insure that the data was really saved.

Table storage

The columns and lines are identified by a key usually created as: page_id + '_' + number. With this scheme, two identical keys are not possible.

The functions to modify the table are:

ABJ + DA storage

Informations about ABJ and DA are stored in one key file per student. The parameters are:

The functions to modify the data are:

add   (login, from, to  , user_name, action_date)
rem   (login, from, to  , user_name, action_date)
add_da(login, ue  , date, user_name, action_date)
rem_da(login, ue        , user_name, action_date)

TT storage

The information about 'Tiers Temps' are stored in a normal table, it contains the student ID, the begin and end date of the exception.

You can add any columns to the table, the informations will be displayed to the teachers, Each column add an information line containing: COLUMN COMMENT: CELL VALUE (CELL COMMENT). This line is not displayed if there is no value nor comment.

To be useful, a single 'Comment' column is needed to explain the case to the teachers.

Referent storage

The information about referents teacher are stored in a normal table named 'orientation_rp'. Each line contains the teacher login, its portals.

The list of students per referent is stored in 'referents_students' table

TOMUSS objects server side

C

Define a cell with the given attributes. It also manage the cell history and a cache to minimize CPU time to generate the minimal JavaScript code for the cell. Attributes of the cell should not be modified directly.

Methods :

Cell

Define a cell with the given attributes. It also manage the cell history and a cache to minimize CPU time to generate the minimal JavaScript code for the cell. Attributes of the cell should not be modified directly.

Methods :

CellEmpty

Define an empty cell. It is only to use less memory than a Cell

Methods :

CellValue

Define a cell without history nor comment It is the most used cell content

Methods :

CellVirtual

The set_ methods in cells returns a new object if needed. It is because an CellEmpty can become a Cell.

Methods :

Column

The Column object contains all the informations about the column. Once the Column is integrated in a table, it memorizes the table pointer.

Methods :

Columns

A set of Column associated to a table. The columns are stored in a list, so they have an index. They also have an unique 'id' used to communicate with the client because the columns are not in the same order in all the clients.

Methods :

Line

The Line object is usable as a Python list.

Methods :

Lines

The Lines object is usable as a Python list of Line. But as it knows the columns it performs some high level functions. It is a dictionnary of lines, the key is the line_id.

Methods :

UE

Contains all the information about the UE (Unity of Evaluation).

Methods :

Types of columns

They are defined in COLUMN_TYPES directory. To add a new column type, the only thing to do is to add 2 new files in COLUMN_TYPES. There is a Python and a JavaScript file per type. Types are defined by a class tree.

The table of types display for each types how to manage the column and the cell. More explanations are in the text.py file.

The attributes of a type are:

attribute_js_valuePython function Translate the Python value of the following attributes into a JavaScript value. No need to modify it
attributes_visiblePython list Column attributes visible for the type. Do not indicate attributes visibles by all the column types, they will be automatically visible.
cell_completionsJavaScript function name Function returning a list of possible completion. It is only used by the Enumeration column type. The completion of the other column type is done using cell_test because there is only one possible completion.
cell_computefunction name Function computing the cell value. The function is defined in Python and translated into JavaScript. You add your own computing function in LOCAL/LOCAL_PYTHON_JS. You compile and check them by running make. You must have regression tests because the translated code is fast but not 100% compatible.
cell_indicatorPython function Function returning the HTML class of the cell in 'suivi' and a value between 0 (bad) and 1 (good). This function is only used to compute student icons.
cell_is_modifiable{0|1} Set to 0 to forbid user modification if the cell contains a number.
cell_testJavaScript function name The function returns the user input cleaned and completed. It may display alerts with alert_append.
formatteJavaScript function name Translate the value in order to display it, for example to round the grades.
formatte_suiviJavaScript function name Same as formatte but the generated HTML may allow to edit the value on the 'suivi' page.
human_priorityint To define the order in the column type chooser.
ondoubleclickJavaScript function name Called on cell double click.
onmousedownJavaScript function name Called on cell selection. No need to modify it
should_be_a_float{0|1} If 1 then the column is displayed by default in the table stat page.
statPython function Returns a dictionnary of statistics about the column. They are used to display stats in the cells of the 'suivi' page or data needed (the possible values in an enumeration).
tip_cellmsgid For the cell tip.
tip_column_titlemsgid For the column title No need to modify it
tip_filtermsgid Indicate an appropriate filter help for the column content
type_type To define the table column in the column type chooser: data, people, ue...
update_allPython function Compute the full column content or of a subset of lines. It is recommended to use the update_all method of 'code_etape.py' and to redefine get_all_values method
update_onePython function Do not use this one because it is too slow to update line per line.
value_rangePython function Returns the range of notation No need to modify it

Attributes of table columns

name  display_table   update_horizontal_scrollbar   update_headers   update_table_headers   need_authorization   formatter  empty  default_value   computed   
type10101function(column, value) { return value ; }function(column, value) { return value === "" ; }Note0
abi_is10001function(column, value) { return value ; }function(column, value) { return value === "" ; }00
abj_is10001function(column, value) { return value ; }function(column, value) { return value === "" ; }00
author00101get_author2function(column, value) { return value === "" ; }1
cell_writable00001function(column, value) { return value ; }function(column, value) { return value === "" ; }= | @ | @=0
columns10001function(column, value) { return value ; }function(column, value) { return value === "" ; }0
comment10101function(column, value) { return value ; }function(column, value) { return value === "" ; }0
completion00000function(column, value) { return value ; }function(column, value) { return value === "" ; }0
course_dates10001course_dates_formatterfunction(column, value) { return value === "" ; }0
delete00000function(column, value) { return value ; }function(column, value) { return value === "" ; }10
empty_is10001function(column, value) { return value ; }function(column, value) { return value === "" ; }0
enumeration00001function(column, value) { return value ; }function(column, value) { return value === "" ; }0
export00000function(column, value) { return value ; }function(column, value) { return value === "" ; }10
fill00000function(column, value) { return value ; }function(column, value) { return value === "" ; }10
freezed10010function(column, value) { return value ; }function(column, value) { return value === "" ; }0
green10000function(column, value) { return value ; }function(column, value) { return value === "" ; }0
greentext10000function(column, value) { return value ; }function(column, value) { return value === "" ; }0
groupcolumn00001function(column, value) { return value ; }function(column, value) { return value === "" ; }0
hidden00000function(column, value) { return value ; }function(column, value) { return value === "" ; }01
import00000function(column, value) { return value ; }function(column, value) { return value === "" ; }10
import_zip00000function(column, value) { return value ; }function(column, value) { return value === "" ; }10
locked00001function(column, value) { return value ; }function(column, value) { return value === "" ; }00
mcq10001function(column, value) { return JSON.stringify(value) ; }function(column, value) { return value === "" ; }[{}, {}]0
minmax10001function(column, value) { return value ; }function(column, value) { return value === "" ; }[0;20] 0
modifiable00011function(column, value) { var e = document.getElementById('t_column_modifiable') ; if ( e ) if ( value >= 2 ) e.style.background = '#F88' ; else e.style.background = '' ; return value ;}function(column, value) { return value === "" ; }00
notation_export00000function(column, value) { return value ; }function(column, value) { return value === "" ; }10
notation_import00000function(column, value) { return value ; }function(column, value) { return value === "" ; }10
position10010function(column, value) { return value ; }function(column, value) { return value === "" ; }0
private00011function(column, value) { return value ; }function(column, value) { return value === "" ; }0
red10000function(column, value) { return value ; }function(column, value) { return value === "" ; }0
redtext10000function(column, value) { return value ; }function(column, value) { return value === "" ; }0
repetition00001function(column, value) { return value ; }function(column, value) { return value === "" ; }00
speak00000function(column, value) { return value ; }function(column, value) { return value === "" ; }0
stats00000function(column, value) { return value ; }function(column, value) { return value === "" ; }10
test_filter10001function(column, value) { return value ; }function(column, value) { return value === "" ; }0
test_if10001function(column, value) { return value ; }function(column, value) { return value === "" ; }0
title00011function(column, value) { return value ; }function(column, value) { return value.substr(0,default_title.length) == default_title && !isNaN(value.substr(default_title.length)) ; }0
upload_max00001function(column, value) { return value ; }function(column, value) { return value === "" ; }50000
upload_zip00000function(column, value) { return value ; }function(column, value) { return value === "" ; }10
url_base00001function(column, value) { return value ; }function(column, value) { return value === "" ; }0
url_import00001function(column, value) { return value ; }function(column, value) { return value === "" ; }0
url_title00001function(column, value) { return value ; }function(column, value) { return value === "" ; }0
visibility00011column_visibility_formatterfunction(column, value) { return value === "" ; }00
visibility_date00011 function(column, value) { if ( value === '' ) return '' ; return column.visibility_date.substr(6,2) + '/' + column.visibility_date.substr(4,2) + '/' + column.visibility_date.substr(0,4) ; }function(column, value) { return value === "" ; }0
weight10001function(column, value) { return value ; }function(column, value) { return value === "" ; }10
width00000function(column, value) { return value ; }function(column, value) { return value === "" ; }40
best10001function(column, value) { return value ; }function(column, value) { return value === "" ; }00
rounding10001function(column, value) { var e = document.getElementById('t_column_rounding') ; if ( e ) e.style.background = (column.type == 'Moy' && value > rounding_avg) ? '#F88' : '' ; return value ; }function(column, value) { return value === "" ; }0
worst10001function(column, value) { return value ; }function(column, value) { return value === "" ; }00

Attributes of tables

name  display_table   update_horizontal_scrollbar   update_headers   update_table_headers   need_authorization   formatter  empty  default_value   computed   only_masters   
masters00101 function(value) { if ( value instanceof Array ) value = value.join(' ') ; else table_attr.masters = value.split(/ +/) ; if ( table_attr.masters.length ) i_am_the_teacher |= myindex(table_attr.masters, my_identity) != -1 ; else i_am_the_teacher = false ; return value ; }function(value) { return value === "" ; }<bound method TableMasters.default_value of <TOMUSS.ATTRIBUTES.tablemasters.TableMasters object at 0x7fe5b0b61710>>00
teachers00101 function(value) { if ( value instanceof Array ) return value.join(' ') ; table_attr.teachers = value.split(/ +/) ; return value ; } function(value) { return value === "" ; }<bound method TableTeachers.default_value of <TOMUSS.ATTRIBUTES.tableteachers.TableTeachers object at 0x7fe5b0b57fd0>>00
abj00000function(value) { return value ; }function(value) { return value === "" ; }00
autosave00001function(value) { return value ; }function(value) { return value === "" ; }100
bookmark00101table_bookmarkfunction(value) { return value === "" ; }100
code00001function(value) { return value ; }function(value) { return value === "" ; }<bound method TableCode.default_value of <TOMUSS.ATTRIBUTES.tablecode.TableCode object at 0x7fe5b0b57cf8>>00
comment00001function(value) { return value ; }function(value) { return value === "" ; }00
contains_users00101table_modifiable_togglefunction(value) { return value === "" ; }100
dates00001date_formatterfunction(value) { return value === "" ; }[0, 2000000000]00
default_nr_columns00001function(value) { return value ; }function(value) { return value === "" ; }000
default_sort_column00001function(value) { return value ; }function(value) { return value === "" ; }000
facebook00000function(value) { return value ; }function(value) { return value === "" ; }00
forms00001function(value) { return value ; }function(value) { return value === "" ; }100
group00001function(value) { return value ; }function(value) { return value === "" ; }00
hiddens00000function(value) { return value ; }function(value) { return value === "" ; }000
hide_empty00100function(value) { if ( value == table_attr.hide_empty ) return Number(value) ; if ( ! table_change_allowed() || ! table_attr.modifiable ) { if ( value != 0 ) change_option('hide_empty', '1') ; else change_option('hide_empty', '0') ; } update_filtered_lines() ; table_fill(true) ; return Number(value) ; }function(value) { return value === "" ; }000
invitation00001function(value) { return value ; }function(value) { return value === "" ; }100
linear00001function(value) { return value ; }function(value) { return value === "" ; }100
mail00000function(value) { return value ; }function(value) { return value === "" ; }00
mails00001function(value) { return value ; }function(value) { return value === "" ; }{}10
managers00001function(value) { return value ; }function(value) { return value === "" ; }<bound method TableManagers.default_value of <TOMUSS.ATTRIBUTES.tablemanagers.TableManagers object at 0x7fe5b0b61208>>00
modifiable00101table_modifiable_togglefunction(value) { return value === "" ; }100
nr_columns00000function(value) { return value ; }function(value) { return value === "" ; }000
nr_lines00000function(value) { return value ; }function(value) { return value === "" ; }000
official_ue00101function(v){return v;}function(value) { return value === "" ; }000
portails00001function(value) { return value ; }function(value) { return value === "" ; }{}10
print00000function(value) { return value ; }function(value) { return value === "" ; }00
private00101 function(value) { if ( (table_attr.masters.length == 0 || ! i_am_the_teacher) && value == 1 && myindex(table_attr.managers, my_identity) == -1 && ! i_am_root ) { Alert("ALERT_colmunprivate") ; return ; } return value ; }function(value) { return value === "" ; }000
rounding10001 function(value) { var e = document.getElementById('t_table_attr_rounding') ; if ( e ) if ( value == 2 ) e.style.background = '#F88' ; else e.style.background = '' ; for(var data_col in columns) { var column = columns[data_col] ; column.need_update = true ; column_attributes.rounding.check_and_set(column.rounding, column) ; } return value ; }function(value) { return value === "" ; }000
statistics00000function(value) { return value ; }function(value) { return value === "" ; }00
t_copy00001function(value) { return value ; }function(value) { return value === "" ; }100
t_create00001function(value) { return value ; }function(value) { return value === "" ; }100
t_export00001function(value) { return value ; }function(value) { return value === "" ; }100
t_import00001function(value) { return value ; }function(value) { return value === "" ; }100
table_delete00001function(value) { return value ; }function(value) { return value === "" ; }100
table_title00001function(value) { return value ; }function(value) { return value === "" ; }<bound method TableTableTitle.default_value of <TOMUSS.ATTRIBUTES.tabletabletitle.TableTableTitle object at 0x7fe5b0b824e0>>00
theme00001function(value) { return value ; }function(value) { return value === "" ; }00
update_content00001function(value) { return value ; }function(value) { return value === "" ; }()00

TOMUSS server side threads

The server need informations from other servers, in order to be responsive, the requests must be performed in threads. To simplify the program, there is a single thread per data type to be updated.

Some external processes launched by crontab compute some files periodically. TOMUSS reread the file is they appear to be modified.

TOMUSS client side

A maximum of work is done by the client, for example, average and other computation are not done by the server.

The table editing page

The client load the page that stay open in order to receive updates as JavaScript codes. The client actions are done by inserting image, the image is inserted where the user interaction took place and at the right of the 'cell' tab. It is done so because, an image must stay visible when the user change of page or filter the table. The image is at the same time the server feedback indicating if the action performed well.

The server, indicate that the action was performed with a javascript code. When the client receive this code, it removes the matching images at the right of the 'cell' tab. In the normal case, there is no image at this place.

If an image is not loaded after some time, the image URL is modified in order to retry the aborted load because navigator do not retry failed loads.

If image loads fails and the server is active, then we assume that communication fails because the ticket is no more valid (IP change for example). In this case, the user is asked to authenticate once more.

Client side objects

As TOMUSS started as a trivial program, it was not developed with JavaScript objects. The only objects are:

A Table object should be added in order to clean the code.

Virtual tables

These tables are not stored into files. They can be :

The 'Display' framework

The 'Display' framework is currently used by the 'suivi' nd 'home' page. It allows to add new informations on the page without modifying code source code. Display trees are defined in PLUGINS/suivi_student.py, or PLUGINS/home3.py. For example, the information about the 'more_on_suivi' plugin is defined as:

from .. import display # The display framework
def display_more_on_suivi(server):
    return configuration.more_on_suivi(server.suivi_login, server)
display.display('MoreOnSuivi', 'BodyRight', 9, data=display_more_on_suivi)

On the javascript side PLUGINS/suivi_student.js:

function DisplayMoreOnSuivi(node)
{
  // To use the data from Login display plugin : display_data['Login']
  // If 'is_a_teacher' is true, it is the teacher view.
  // To hide the block from screen: return ''
  return node.data ; // Display the server computed content without change

  // It is possible to return values to insert in the container-DIV attributes
  //   return ["HTML to display", [classes...], [styles...], "other attrs"]
  // see DisplayCellBox for an usage example
}
// Defines the data needed from other display plugin.
// The plugin will be called once data is retrieved from server.
// The default value for this function is itself: ['MoreOnSuivi']
// It is possible to wait data from multiple plugins:
//      DisplayMoreOnSuivi.need_node = ['MoreOnSuivi', 'Login'] ;
// If no data is needed, use an empty required list: []

On the CSS side PLUGINS/suivi_student.css:

.MoreOnSuivi { background: white }

The node object has following attributes: name, children, containers, priority.

On the javascript side, a display function can call other display function multiple times. For example DisplayUEGrades call CellBox to display each grade.

TOMUSS Translation

If the user has not defined its language in its preferences, it is the browser languages that is taken. The server default language is used to created table for column names, comments, value of literals as YES, NO...

The translations are in TRANSLATIONS and LOCAL/LOCAL_TRANSLATIONS directories. The LOCAL translation override the default translation.

some message ID are not in the source code because they are computed. for example for the column types and the attributes.

Message ID format is usually in this form: TYPE_PLUGIN_ID where TYPE is

BEWARE: if you change the language of the server, you need to restart it because it is complex to change it live.

TOMUSS Administration

Semesters

TOMUSS retrieves the current student list without knowing to which semester it applies.

The procedure to change of semester is the following:

WhenWhat
You want to use the next semester In your local configuration file, add the new semester with a new port number :
suivi.add(2009, 'Automne' , socket.getfqdn() + ':%d', 8891)
You can indicate the same port number for multiple semesters, in this case, only one process is launched for these semesters.
Set the next semester name in 'year_semester_next' in the TOMUSS configuration table.

The tables are editable in both semesters. But current student lists do not appear in the next semester.

Il you want to update student lists for the next semester, add it to year_semester_update_student_list in the configuration table.

New lists of students are accessible.

Set 'year_semester' to the same value than 'year_semester_next'.

Copy the 'referents_students' table in the new semester.

Restart TOMUSS: make stop ; make

Il you want to continue update student lists for in the old semester, add it to year_semester_update_student_list in the configuration table.

Il you want to allow the previous semester tables to be editable, add it to year_semester_modifiable in the configuration table.

If the table of the previous semester must be modifiable by default, you can indicate it in the configuration table in the modifiable semester list. The current and next semester are modifiable by default.

Table templates

Table templates are stored in 'TEMPLATES' directory. When loading a page, if the file match a template it is applied, if the semester match a template it is applied. Only one template can be applied. The templates may defines :

Semester templates defined are:

Table templates defined are:

The table template must be created in order to match the teaching organization.

Do not store information in the module itself because it will be lost when the TEMPLATE is reloaded (it is triggered by a file change).

The recommended way to define columns in a template is the table method update_columns. It will create and update columns definitions.

If you have configuration parameters associated to your Plugin/Template it is a good idea to use 'utilities.Variables' in order to associate the values to the TOMUSS table '0/Variables/template_name' So the user can change the configuration at running time. You can give access rights to this table to non-root users.

Authentication process

The URLs start by a ticket. If there is no ticket, the navigator is redirected to a service in order to get a ticket. If CAS is not used, then a random ticket number is generated.

The tickets are ``exchanged´´ between the TOMUSS processes as a python module containing all the valid tickets.

To be valid, a ticket must be used with the same navigator and the same IP than the first time.

'ticket.py' define the ticket object that store tickets and parse URLs. It is a generic and not configuration dependent.

A new ticket can be used to revalidate an old ticket, it is useful if the client IP changed.

'authentication.py' can be redefined by a local plugin in order to have the good code for the functions 'ticket_login_name' and 'ticket_ask'. If CAS is not used, then these functions assume that the authentication is done with Apache/.htaccess/.htpasswd with the Basic authentication method (the clear password is checked with su)

In regtest mode, the URLs accept =username as a valid ticket for the user.

Apache/NGINX configuration

If you have multiple suivi servers, then the configuration must be updated when there is a new suivi server. If you have only one suivi server for all the semesters, then no change is needed.

TOMUSS urls are for example:

To have TOMUSS working with nicer URLs we can configure Apache as:

<VirtualHost tomuss.fr:80>
ServerName tomuss.univ-lyon1.fr
RewriteEngine On
# It is the web server (not TOMUSS) that send the static files (UNTESTED)
RewriteRule ^/files/([0-9.]+)/(.*)  /home/tomuss/TMP/$1/$2 [L]
RewriteRule ^(.*) http://tomuss1.fr:8888$1 [P]
</VirtualHost>

<VirtualHost tomusss.fr:80>
ServerName tomusss.univ-lyon1.fr
RewriteEngine On
RewriteRule ^(.*/2009/Printemps/.*) http://tomuss3.fr:8890$1 [P]
RewriteRule ^(.*/2008/Automne/.*)   http://tomuss2.fr:8889$1 [P]
# 'Suivi' on the current semester
RewriteRule ^(.*)                   http://tomuss2.fr:8889$1 [P]
</VirtualHost>

We can configure NGINX with:

proxy_buffering off;         # To be really interactive
proxy_read_timeout 1000000;  # To keep connection open

server {
    listen   80;
    server_name  tomuss.fr;
    # It is the web server (not TOMUSS) that send the static files
    location ~ ^/files/ {
      gzip on;
      gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript text/x-js;
      root /home/tomuss/TMP ; # Must be readable by NGINX
      add_header Cache-Control "max-age=86400";
      rewrite  ^/files/([0-9.]+)/([^/]*) /$1/$2 break ;
    } 
    location    /                       { proxy_pass http://tomuss1.fr:8888; }
}

server {
  listen   80;
  server_name  tomusss.fr;
  location    /         { proxy_pass http://tomuss3.fr:8890/2009/Printemps/; }
  location ~ ^/(=[^/]*/)?2009/Printemps { proxy_pass http://tomuss3.fr:8890; }
  location ~ ^/(=[^/]*/)?2008/Automne   { proxy_pass http://tomuss2.fr:8889; }
}

With this, the URLs become:

TOMUSS configuration

The initial TOMUSS configuration is stored in configuration.py. DB is the production database name.

configuration.py must not be modified because it will be overwrited by each TOMUSS release. The file LOCAL/config.py must be edited to customize and configure TOMUSS.

This customization is done by replacing default functions by yours. For the UCBL university there is:

# The login and the student ID are not the same at the UCBL
# login_to_student_id, the_login, login_to_id (JavaScript)
from . import student_id

# get_ue_dict: Retrieve all the informations about all the UE
from . import spiral

if not regtest:
    # ticket_login_name: Get login name from ticket
    # ticket_ask: redirect the browser to ask a ticket
    from . import auth

    # stupid_password: Returns True if the password is stupid
    from . import checkpassword

    # students: Iterator on all the students of an UE, returns :
    # (student_id, firstname, surname, mail, group, sequence)
    from . import students_of_ue

Most of the configuration values are modifiable while TOMUSS is running by editing as 'root' the table named 'http://......../0/Dossiers/config_table' The values in this table have precedence over values stored in the configuration source files. Nevertheless the first time, this table is created using the configuration source files

TOMUSS authorizations

The home page of root user contains a link on the plugin configuration table. For each plugin, you can indicate a list of allowed item :

It is recommended to use group name because most of the time, a few plugins have the same users allowed. The user groups are defined in the config ACLS table. On the first column you enter an item as in the plugin table, on the second column, you indicate to which group you add the item.

If a group contains items prefixed by '!', then they are evaluated first, and if the user is contained in the group, it will be rejected. There is an exception: if the group or list contains only one '!' item, then it returns True if the user is not in the group.

Example displayed as a tree, the '!' is not part of the group name:

staff
  * ldap:GROUP_X          # Evaluated after !, so some user may be rejected
  * !FORBIDEN             # Define the list of forbidden users
     * ldap:GROUP_Y
     * !USER_A            # It will not be in FORBIDDEN
     * !grp:teachers      # No teachers in FORBIDDEN
  * teachers
     * ldap:GROUP_T
     * USER_B
  * administratives       # FORBIDDEN is tested before administrative
     * ldap:GROUP_A       # If users are in GROUP_Y they will not
     * USER_C             # be in 'staff' group (even USER_C)

TOMUSS Regressions Tests

The makefile goal regtest run an infinite loop on some server tests. The loop is broken if there is a problem.

The URL http://SERVER/2009/Test/javascript_regtest_ue run some JavaScript tests on the user interface. These tests works on FireFox and Opera but not on IE.

TOMUSS starting and stopping

The makefile goals start and stop allow to manage the TOMUSS services. There is one database modification service and one 'suivi' service per semester in order to not have huge processes. The 'suivi' processes are huge because they load the full semester in memory.

UPDATE SINCE Version 5.2.0: once students are indexed by SCRIPTS/bilan.py the 'suivi' process do not load complete semester but only needed tables. In this case only one 'suivi' process is needed. To use only one process, use the same port for all the semesters.

TOMUSS install on production server

The goal install of the Makefile runs the script install that do all the work to replace a running TOMUSS by the new release. IT IS A VERY BAD IDEA TO RUN THIS SCRIPT WITHOUT UNDERSTANDING IT.

The symbolic link DB points on directory where the database is stored.

Required packages : python-ldap, python-imaging, gzip, inkscape, gettext

Recommended packages : gnuplot, graphviz, rsync (distant mirroring), catdoc (csv extract from xls)

TOMUSS managing

The TOMUSS root can use the following features:

Some non automatic work to do: see LOCAL/Makefile

TOMUSS Pitfall

When loading a Python module, never store the module attributes in local variables because they can change on module reload or if the tomuss startup is not terminated.

Never modify the database files if the table is loaded in the TOMUSS server. If you must really do it:

How to customize the display

The table display can only be modified by patching middle.js except for the information tab that is filled with table_info content.

The suivi and home page are defined dynamically and can be easily enhanced by adding content boxes. Activate the developer helps to explore the content tree and get the name and display priority of the items. The content tree is defined in home3.py and suivi_student.py and is modifiable by any module.

Python side:

from .. import display

def myData(server):
    return [1, ['python', 'data'], 'structure']

display.Display(
   "YourBoxName",   # DisplayYourBoxName(node) JS function returns the HTML
   "ContainerName", # Use a list if there is more than one container
   float(priority), # Can be negative
   # js="Vertical", # Use DisplayVertical to display children
   # data=myData    # The fast functions are evaluated first
   )

JavaScript side:

function DisplayYourBoxName(node)
{
  // node.data is the value returned by 'myData'
  return node.data.join(' === ') ;
}
// DisplayYourBoxName.need_node = ['YourBoxName'] ;

To store cleanly your JS/CSS code:

from .. import files

files.add('LOCAL', 'my_boxes.js')
files.files['home3.js'].append('my_boxes.py', files.files['my_boxes.js'])

files.add('LOCAL', 'my_boxes.css')
files.files['home3.css'].append('my_boxes.py', files.files['my_boxes.css'])

To modify the student suivi page, replace home3 by suivi_student

When debugging JS and CSS only a page reload is needed. When modifying 'myData' function, the plugins must be reloaded to take the change into account.

Functions to redefine in order to customize TOMUSS

If you want your TOMUSS customization to not be destroyed by a version change you must follow the procedure. The functions listed are the ones that you need to modify, but you can modify any function you want.

In the following table the JavaScript functions must not be modified in the JavaScript source, they can be redefined by LOCAL/config.py script using this procedure:

import files

files.files['lib.js'].append("a_key", """
function the_function_to_be_redefined()
{
}
"""

The Python function must not be modified in the Python sources, they must be redefined in LOCAL/config.py script using this procedure:

import a_module

old_one = a_module.to_be_redefined

def to_be_redefined():
   old_one()

a_module.to_be_redefined = to_be_redefined
./abj.pydef prune_abjs(abj_list, group, sequence, ue_code):
    """Trim unecessary ABJS"""
    return abj_list
When a student has an ABJ it may be outside of UE schedule. The function returns the ABJs without ABJ outside of UE planning.
./configuration.pydef semester_span(year, semester):
    try:
        i = semesters.index(semester)
    except ValueError:
        return
    def p(month):
        overflow = int(month > 12)
        return '%d/%d' % (month - overflow*12, year + overflow)
    return '1/%s 31/%s' % (p(semesters_months[i][0]),
                           p(semesters_months[i][1]))
Time span of the given semester
./configuration.pydef display_this_semester_to_the_student(year, semester, login):
    return True
Does the student may see this semester on its 'suivi' page?
./configuration.pydef is_an_official_ue(code):
    "Must be 0 or 1 (JavaScript compatible)"
    return 1
Return True if the name is the code of an official UE. The official UE are displayed to the students in the suivi.
./configuration.pydef is_a_portail(ldap_group):
    if ou_portail_contains in ldap_group:
        return ldap_group
If the LDAP group is a portail, the returns the portail name. Else return None
./configuration.pydef get_managers(ue_code):
    "Return a list of managers logins"
    return []
Return the list of table manager Their only right is to modify the 'masters' table attribute
./configuration.pydef is_a_student(login):
    return login and len(login) >= 2 and login[1].isdigit()
Return True if is is a student login
./configuration.pydef first_registration(login):
    from . import inscrits
    if login[1:3] == str(year_semester[0])[2:4]:
        for group in inscrits.L_batch.member_of_list(login):
            if '1A,OU=' in group:
                return 1
            if '2A,OU=' in group:
                return 2
            if '3A,OU=' in group:
                return 3
    return 0
Returns 0 if it is not the first registration of the student. Else, returns the year of the first registration.
./configuration.pydef student_in_first_year(login):
    fr = first_registration(login)
    if fr == 1:
        return True
    if isinstance(fr, int):
        return False
    return None
Returns True if the student is in the first year. False if it is not Node if the information is not known The information is displayed in the 'blocnote'
./configuration.pydef concerned_teachers(server, the_student):
    if server.ticket.user_name == the_student:
        return True
Returns True if the teacher can see a private suivi.
./configuration.pydef student_inscrit_value(table, line):
    return ''
Returns HTML class names for table lines class in the table editor. So the line style can be computed from any information about the student. Line style 'ok', 'non' (unregistered), 'tierstemps' are computed in a static way in server or client side. If this function returns 'italic_student', you can add to 'style.css' : .italic_student { font-style: italic }
./configuration.pydef student_class(login):
    return ''
Returns HTML class name for the refered student line on the home page
./configuration.pydef cell_change(table, page, col, lin, value, date):
    return
Function called on user cell change by TEMPLATES/_ucbl_.py:cell_change and so in Printemps and Automne template.
./configuration.pydef do_not_update_student_list(table):
    return False
Return True if the table student list must not be updated even if it should be. For example to not update student lists between 17:15 and 18:15
./configuration.pydef display_tree(server):
    if server.suivi_login == 'k01':
        # For 'k01' UE2 and UE3 are displayed inside UE1 on the suivi page
        return {"UE-UE2": "UE-UE1", "UE-UE3": "UE-UE1"}
    return {"UE-UE5": "UE-UE4", "UE-UE6": "UE-UE4",
            "UE-UE7": "UE-UE5", "UE-UE8": "UE-UE5"}
Returns the UE tree for the student, for each UE its parent UE:
./configuration.pydef preferences_change(login, key, old_value, new_value):
    pass
Hook called each time a preference is updated by the user.
./configuration.pydef local_columns(table):
    # Must be starting with 0_6 key
    # Column order is the alphabetical sorting of keys
    some_columns = {
        '0_6': {'title':'UE', 'comment': "UE d'origine de l'étudiant",
                'type':'Text', 'width':4},
        '0_7': {'title': 'Horaire', 'comment': "Horaire de l'enseignement",
                'type':'Text', 'width': 5},
        '0_8': {'title':'Information','comment':"Informations complémentaires",
                'type':'Text', 'width': 6},
        '0_9': {'title': 'Note_Semestre_Avant_Jury', 'comment': "Note de l'UE",
                'type': 'Note', 'width': 3},
        '0_a': {'title': 'Remarques', 'comment': "Commentaire de l'enseignant",
                'type': 'Text', 'width': 8},
        }
    if table.ue.startswith('SP-'):
        return some_columns
    if table.ue.startswith('TS-'):
        # Should be defined the LOCAL directory
        table.table_attr(table.pages[0], 'default_nr_columns', 9)
        del some_columns['0_8']
        return some_columns
    return {}
Returns a dictionary of column definition to add to the standard student tables.
./configuration.pydef picture(student_id, ticket):
    from . import utilities
    return ticket.url(utilities.StaticFile._url_+'/picture/'+student_id+'.JPG')
This function returns the URL of the student picture. This example assumes that TOMUSS itself send pictures.
./configuration.pydef external_bilan(login):
    return "[]"
To add external information in the 'bilan' This function is used by PLUGINS/bilan.py to get external information
./configuration.pydef suivi_check_student_lists(login):
    return True
Returns True to display the UE name if the student is registered for the UE but is not in the TOMUSS table (because it is uncreated for example)
./configuration.pydef home_page_js_hook(dummy_server):
    return ''
This function returns JavaScript data, for example 'var my_var = "Value";'. The data is used by 'generate_home_page_hook' javascript function that is called when the home page HTML is generated.
./configuration.pydef table_redirection(server):
    from . import document
    if os.path.exists(document.table_filename(server.the_year,
                                              server.the_semester,
                                              server.the_ue)):
        return
    if server.the_ue.startswith('EC-'):
        server.the_file.write("<script>window.location = '%s%s' ;</script>"
                              % (server_url,
                                 server.path.replace("/EC-", "/UE-")))
        return True
When opening a table, this function can trigger a redirection based on the argument analysis
./configuration.pydef date_to_time(date):
    """XXX Convert a french date to seconds"""
    return time.mktime(time.strptime(date, "%d/%m/%Y"))
def tuple_to_date(time_tuple):
    """XXX Convert time tuple to date"""
    return time.strftime('%d/%m/%Y', time_tuple)
These functions must NEVER be redefined once TOMUSS is in usage, because the date are stored in plain text in files. So changing this function will broke every existing files
./referent.pydef port():
    return configuration.the_portails['UFRFST'] 
List of LDAP OU of student to be affected to a 'referent'
./referent.pydef not_in():
    return ()
List of LDAP OU of student not to be affected to a 'referent'
./referent.pydef need_a_referent(login):
    """To be redefined"""
    return True
Return True if the student need a referent teacher
./referent.pydef need_a_charte(login):
    return False
Return True if the student must sign an interactive document
./referent.pydef student_list(f, pportails, not_in_list):
    f.write('<h1>' + utilities._("TITLE_referent_student_list") + '</h1>\n')
    students = {}
    print(pportails)
    for portail in pportails:
        f.write('<h2>' + portail + '</h2>')
        f.flush()
        for sstudent in inscrits.L_batch.member_of(portail):
            f.write(sstudent[0])
            for p in sstudent[4]:
                if p in not_in_list:
                    break
            else:
                # Not in any forbidden list
                f.write(' ')
                f.flush()
                s = Student(sstudent, portail=portail)
                if not s.licence:
                    f.write('(!L) ')
                    continue
                if len(s.ues) == 0: # Aucune IP !!!!
                    f.write('(!IP) ')
                    # XXX continue
                students[sstudent[0]] = s
                continue
            f.write('(L3) ')
Compute the student list needing a referent
./referent.pydef analyse_groups(student, groups):
    # Update list of the UE of the student
    for ue in groups:            
        if ue.startswith(configuration.ou_ue_starts):
            student.ues.append(ue[6:].split(' ')[0])
This function computes for the student some attributes needed to assign it a referent teacher. Attributes are : discipline (set), ues (list), licence, licence_first_year
./referent.pydef remove_student_from_referent_hook(referent, student_id):
    return
Triggered when a student is removed from a teacher
./referent.pydef add_student_to_referent_hook(referent, student_id):
    return
Triggered when a student is added to a teacher
./referent.pydef search_best_teacher_local(student, sorted_teachers, f, all_teachers):
    if 'INFO_L3' in student.discipline and student.primo_entrant:
        return all_teachers['elodie.desseree']
This function returns the teacher to assign to a student. It is only needed for special case of the generic algorithm.
./referent.pydef student_need_a_referent(student, all_cells, debug_file):
    return True
Returns True if the student need a referent. This function can heavely modify 'all_cells' and other data to make some adjustement.
./referent.pydef teacher_keep_student(tteacher, student_id):
    return False
Returns True if teacher want to keep the student even if the student must not have a referent teacher
./utilities.pydef the_login(student):
    return safe(str(student))
If the student login in LDAP is not the same as the student ID. This function translate student ID to student login. The returned value must be usable safely.
./utilities.pydef stupid_password(login, passwords):
    return False
This function returns True if the user uses a stupid password. Potential stupid passwords are in the 'passwords' list. Each of the passwords should be tried to login, if the login is a success, the password is bad.
./document.pydef table_head_more(dummy_ue):
    return ''
This function returns javascript code to be included in the header of the 'ue' table
./teacher.pydef get_ue_dict():
    if configuration.ldap_server[0] != 'ldap1.domain.org':
        ldap_ues = inscrits.L_batch.get_ldap_ues()
    else:
        ldap_ues = ()
    ues = {}
    for apoge in ldap_ues:
        ues[apoge] = UE(apoge, [], 'UNKNOWN TITLE',
                        [], 0, [], 1, 1, [], 0)
    for i in range(1,10):
        name = "UE%d" % i
        ues[name] = UE(
            name, # UE Name
            ['ue%d.master' % i],# Teacher names
            'UE%d Title' % i,  # UE title
            [],                 # Departments of UE
            1000+i,             # SPIRAL key for the UE
            ['ue%d.master' % i],# Login of teachers
            1,                  # #students registered
            0,                  # #students registered in EC
            ['ue%d_test@test.org' % i], # Teachers mails
            0,                  # ADE key for the UE
            ) 
    return ues
Returns the UE list from LDAP or other sources. This function is called once per night, so it can be long to execute. The function result is stored as a Python and JavaScript file
./authenticators.pydef password_is_good(login, password):
    """Check clear text password"""
    if PAM:
        return password_is_good_PAM(login, password)
    else:
        return password_is_good_su(login, password)
Return True if the user password is good
./plugins.pydef plugins_tomuss_more():
    pass
This function do the import of LOCAL Plugins for the TOMUSS server
./plugins.pydef plugins_suivi_more():
    pass
This function do the import of LOCAL Plugins for the 'suivi' server
./inscrits.pyclass LDAP(LDAP_Logic):
This class can be replaced by a subclass of itself in order to replace the 'students' method returning the student list for an UE. The student list may be computed without using LDAP.
./inscrits.pydef login_to_student_id(login):
    return utilities.safe(login)
If the student login in LDAP is not the same as the student ID. This function translate student login to student ID. The returned value must be usable safely.
FILES/utilities.jsfunction student_picture_url(login)
{
  if ( login )
    return add_ticket('picture/' + login_to_id(login) + '.JPG') ;
  return '' ;
}
This function returns the URL of the student picture.
FILES/utilities.jsfunction student_picture_icon_url(login)
{
  if ( login )
    return add_ticket('picture-icon/' + login_to_id(login) + '.JPG') ;
  return '' ;
}
This function returns the URL of the student picture icon. 30 pixel wide
FILES/utilities.jsfunction local_number(n)
{
  if ( server_language == 'fr' )
    return n.replace('.',',') ;
  else
    return n ;
}
This function translates a english formatted number into the local format. Currently only used by column export.
FILES/utilities.jsCurrent.prototype.missing_id = function(value)
{
  return (this.data_col !== 0
  && lines[this.line_id][0].is_empty()
  && value !=='') ;
} ;


Current.prototype.input_div_focus = function()
{
  if ( this.focused )
    this.input_div.style.border = "3px solid blue" ;
  else
    this.input_div.style.border = "3px solid gray" ;  
} ;

Current.prototype.change = function(value)
{
  this.input_div_focus() ;
  
  if ( this.blur_disabled )
    return ;

  hide_the_tip_real(true) ; // To hide the enumeration menu

  // Save modification in header before moving.
  if ( element_focused !== undefined )
    {
      if ( element_focused.blur )
element_focused.blur() ;
    }
This function returns true if the student ID is missing on the line. It is called for each interactive cell change. If 'true' is returned, an alert is displayed to the user.
FILES/abj.jsfunction is_a_student_login()
{
  var v = student.value.replace(/[^a-zA-Z0-9-_.]/g, "");
  return v.length == 8 ;
}
Return true if it is a student login
FILES/lib.jsfunction login_to_id(login)
{
  return login ;
}
If LDAP students login and real students ID are not equals then a translation must be done. This function translate the login to a student number.
FILES/lib.jsfunction the_login(login)
{
  return login ;
}
If LDAP students login and real students ID are not equals then a translation must be done. This function translate the student number to a login
FILES/lib.jsfunction do_change_abjs(m)
{
  the_student_abjs = m ;
}
FILES/middle.jsfunction modification_allowed_on_this_line(line_id, data_col, value)
{
  if ( value === '' )
    return true ; // allowed to erase cell value on any line
  if ( myindex(semesters, semester) == -1 )
    return true ;
  if ( tr_classname === undefined )
    return true ;
  if ( ! popup_on_red_line )
    return true ;
  if ( lines[line_id][tr_classname].value == 'non' )
    return true ; // Returns false here to forbid red line editing
  return true ;
}
Some table lines must not be modified. This function return 'true' to allow the line editing.
FILES/middle.jsfunction update_student_information(line)
{
  update_student_information_default(line) ;
}
Template can redefine this function.
PLUGINS/picture.pydef get_the_picture(student_id, filename):
    return filename
configuration.get_the_picture = get_the_picture
configuration.icon_picture_width = 30
configuration.picture_ttl = 24*3600
configuration.picture_extension = '.JPG'
Read/compute the student picture from a database. Store it in the filename indicated if it needs to be cached. Returns the filename containing the picture to display
PLUGINS/timeline.jsfunction timeline_display_external_info(year, semester, code, bilan)
{
  return code ;
}
Display the UE name in the timeline
PLUGINS/suivi_student.jsfunction DisplayUETreeHook(node)
{
}
Called before UE tree construction
PLUGINS/suivi_student.jsfunction get_ue_priority(ue)
{
  if ( ue.cached_priority )
    return ue.cached_priority ;
  var txt = [ue.ue.replace(/-[1-9]*$/, '')] ;
  if ( txt[0] != ue.ue )
      txt.push(99999 - get_nr_real_values_in_the_line(ue)) ;
  else
      txt.push(90000) ; // Priority to the table without extension
  for(var i in ue.line)
    txt.push(ue.line[i][0]) ;
  ue.cached_priority = txt.join('\001') ;
  return ue.cached_priority ;
}
This function allow to sort the table on the suivi page in the wanted order. By default the UE code and the line content if equality.
PLUGINS/bilan.jsfunction update_ues_with_external_data(resume, external_info)
{
  var ues = [] ;
  for(var i in resume)
    ues.push(i) ;
  ues.sort(function(a,b) {
     if ( resume[a][0] > resume[b][0] )
       return 1 ;
     if ( resume[a][0] < resume[b][0] )
       return -1 ;
     return 0 ; } ) ;
  return ues ;
}
Update the'ues' list with external informations. 'ues' is a dictionary of tables (see SCRIPTS/bilan.py). This function returns the order of the lines to display.
PLUGINS/bilan.jsfunction bilan_external_header(login)
{
    return '' ;
}
Returns a string to insert after the title of the bilan page.
ATTRIBUTES/columnimportzip.jsfunction get_the_upload_url()
{
  return url ;
}
Send the url allowing to jump over a web frontend not allowing big files. It is only used for importing zip file. It is not used for the student file uploading.