------------------------------------------------------------------- HEAD ------------------------------------------------------------------- Added optimize funciton to dpp and to dependent libs Add -XX:+HeapDumpOnOutOfMemoryError When a bad XML document is uploaded and raises a SAX parse exception, the bad file is written to disk and the message returned to the uploaded contains the name of the bad file. This can be used as input to a support call and error tracing so that support can identify and debug the file in error. once marked as deleted, no operation can cause a record to be re-indexed into the solr repository - Mantis issue 348 -- Primary development moved to v3 -- This version for patching and maintenance releases ------------------------------------------------------------------- ISPP 0.11 28-SEP-2009 ------------------------------------------------------------------- Changed solr locking config from simple to single Changed session time out to 60 minutes - This should help soften problems when server clocks are incorrectly configured. Resolved mantis issue 156 http://bugs.k-int.com/view.php?id=156 Range searching now uses inclusive for >=,<= exclusive for < and > Resolved mantis issue 159 http://bugs.k-int.com/view.php?id=159 always add echoedSearchRetrieveRequest element Resolve issue 140 - soap upload not correctly checking permissions of upload user against record owner. ------------------------------------------------------------------- ISPP 0.9 20-Feb-2009 ------------------------------------------------------------------- Added util for searching gazeteer on spatial page Fixed problem searching for ispp.postcode/location. Mantis 160 Added explicit error if SRU version parameter not supplied. - mantis 158 Now validate all controlled terms, rather than just ones for which we have been supplied an controlled vocab by the project - mantis 139 Corrected School Pickup XPath from Pickup[HasProvision='true'] to Pickups[HasProvision='true'] Added text to provider search page to make it explicit when % wildcards are available for DB search and * can be used for solr. Added default search for * in dc.title. Resolved the following issues from mantis issue 0000103 - "Issues with the GUI front End" 1. Solr Index Administration Page: resoring --> resorting 2. Asked for more info, don't understand really. 3. Navigatin Links should match page name - Done the ones i can find. Please raise individual issues for each instance which is still wrong. 4. Data Provider Administration link takes user to a Data Source Administration Page --> please change Page Heading 5. Restore Indecies link takes user to Solr Index Administration - please ensure heading and link match 6. Upload page: The deposited item can be accessed here Not document ID is available. --> The deposited item can be accessed here, there is no document ID available. Probably best we fix our terminology and refer to "Data Providers" and "User Accounts" there's no such thing as "Data Provider Account", 2 separate things (Even if they often relate to 1 organisation) 7. Able to create a data source account without any text entered within the 'data source identifier' field or with just a space 8. Able to create a data source account without any text entered within the 'Source Name' field or with just a space 9. When creating a data provider account the Username provided is placed in the username field as expected but also displayed in the personal name Added caters for over 8's flag - Note for next release In clustered environments, running the following command on each SOLR slave host will cause the right config files to be automatically copied from mastersolr to the client solr index. This saves manually copying the slave_config.xml on the master to the confi.xml on each slave. The command needs to be run against each slave node in the network. http://slave_host:port/solr/replication?command=snappull&masterUrl=http://mastersolr:8080/index Full details here : http://wiki.apache.org/solr/SolrReplication ------------------------------------------------------------------- ISPP Aggregator Interim Release by II (Not via release procedures) 03/02/2009 ------------------------------------------------------------------- Below are the access points from robs document, the mapping in solr, and the mapping through the SRU mappings.properties file. I have extracted and summarised the information below from the index_mapping.properties which andy has patched in the past to add extra properties. 1) Simple Search - In solr "text:searchterm" (existed already) - In SRU "searchterm" (existed already) 2) ISPP Subject - In solr - dc.subject:subjectkw (existed already) - In SRU - dc.subject=subjectkw already existed - In SRU - "Subject" Added in this release 3) Location - The spec document shows that we clearly don't have a shared understanding of the way I thought we agreed place search will work. All the values listed in the xpath should be capable of matching place searching, but it does not match the literal strings from the document. What happens is : Postcode or other individual place specifier is extracted from the input document. The X,Y lat/long are encoded in the solr result record. When a user does a spatial search, their input term is converted into an X,Y and then all records close to the input X,Y are returned, ranked by distance. - In solr - ispp.postcode (existed already) - In SRU - ispp.postcode (existed already) - In SRU - Location (Added in this release) 4) Age Suitability - In solr - ispp.age_min and ispp.age_max (existed already) - In SRU - ispp.minage and ispp.maxage (existed already) - This release Added SRU aliases AgeRangeMin and AgeRangeMax 5) Eligibility - In solr - ispp.eligibility (Already existed) - In SRU - ispp.eligibility (Already Existed) - In SRU - EligibilityCriteria (Added in this release) 6) ReferralCriteria - In solr - referral_criteria_s (Added in this release) - In SRU - ReferralCriteria (Added in this release) 7) AccessChannel - In solr - mode_of_access_s (Added in this release) - In SRU - AccessChannel (Added in this release) 8) LanguageSpoken - In Solr - language_spoken_s - In SRU - LanguageSpoken 9) WheelchairAccess 10) SpecialNeeds 11) CulturalProvision 12) SpecialDiet 17) DisabledAccess 23) SchoolPickUp - As discussed with Rob and others, global binary indexes are a poor way to implement this kind of Flag and are also not extensible. - As agreed, all mapped to - In SOLR - Flags (Already exists) - in SRU - Flags (Already exists) - flags:WheelChairAccess flags:SpecialNeeds flags:CulturalNeeds flags:DietaryNeeds flags:SchoolPickup flags:WheelChairAccess (Values added in this release) 13) Accreditation - in SOLR - accreditation_s (Added in this release) - In SRU - Accreditation (Added in this release) 14) Facilities - In SOLR - ispp.facilities - In SRU - ispp.facilities (Already Exists) and Facilities (Added In This Release) 15) ReferralProcedures - In SOLR - referral_procedures_s (Added in this release) - In SRU - ReferralProcedures (Added in this release) 16) Travel - In SOLR - travel_information_s - In SRU - TravelInformation 18) LeisureCardInformation - In SOLR - leisure_info_s (Added in this release) - In SRU - LeisureInformation (Added in this release) 19) WaitingList - In SOLR - waiting_list_s (Added in this release) - In SRU - WaitingList 20) CRB - In SOLR - crb_policy_s (Added in this release) - In SRU - CRB (Added in this release) 21) ChildcareType - In SOLR - childcare_type_s (ADded in this release) - In SRU - ChildcareType (Added in this release) 22) ProvisionType - In SOLR - provision_type_s (Added in this release) - In SRU - ProvisionType 24) Over8s - Couldn't see why this can't be done with AgeRange? 25) Vacancies - In SOLR - immediate_vacancies_s (Added in this release) - In SRU - Vacancies (Added in this release) 26) National - In SOLR - national_s (Added in this release) - In SRU - National 27) Last Updated - There is a standard solr property for this, pls see SOLR docs 28) ProviderName - In SOLR - provider_name_s - In SRU - ProviderName 29) Distance - There is no "Distance" index, it's a function of the search (There can't be a property, it's relative to the input search term?) - Fixed error when trying to auth/deny a priv request to an invalid email address. error is given for information now but auth/deny proceeds. - Fixed an error on user home page when there are no datasources available. - Alterred index restore to allow redeposit of any xml instance - dpp indexing host target changes to mastersolr from localhost - for replicated setup. You will need to add a hosts entry! ------------------------------------------------------------------- ISPP Aggregator 0.4 09th Dec 2008 ------------------------------------------------------------------- - Correctly expire login cookie - error 74201 We've investiagted this and if you use a cookie monitor you will see that the session cookie is removed and replaced by an auauthorised one. However, when you click "login" the browser automatically sends saved authentication credentials and causes the login. This is tested by selecting nothing but "clear saved authentication" in the clear private data dialog. Problem is a browser one. Looking for other work arounds to get desired behaviour. Currently only option is browser config tho. - Extra error checking added arounf user permissions dialog - 74877. Retest please. - Unable to reproduce MSIE bug 74581. Please retest and supply full IE version numbers. - Text on user home page 75907 - Fixed. Combo holds both identifier and name of provider. - Resolved 75883 - Problem with local DNS timeout. Nothing aggrgeator can do until DNS time out. - Deduplication Setup - 75892 - Configured two new fields dd_title and dd_postcode. These will be present in SOLR indexes. Dupliocate marking will be configured once I've seen some real test data populating these fields and had a chance to review - Action by tribal : K-int need real test data before this can proceed. ------------------------------------------------------------------- ISPP Aggregator 0.3 28th Nov 2008 ------------------------------------------------------------------- - Add workround for MISE bug not sending correct (x)html mimetype in the Accepts header for resource upload. - Added SAX XML Parsing error output for document upload. Controlled by the service applicationContext.xml config block. If aggregator.validation.mode is set to strict then warnings/errors/fatals will be reported back to the UI. Where possible, document processing will continue. - High level error categories moved into messages object, and classified as follows: -- INTERNAL_ERROR = 101; -- INTERNAL_NO_HANDLER_SCHEMA = 102; -- INTERNAL_NO_HANDLER_MIME_TYPE = 103; -- INTERNAL_NO_CONTENT_TYPE = 104; -- PARSE_ERROR = 201; -- NO_PERMSSION = 301; - Error structure changed to be a list of the above error codes and additional information messages. Returned for both rest and soap upload - Fixed error in SRU response where textual data was incorrectly being unescaped. - Altered display of "On Behalf Of" combo in upload page to include both identifier and name of provider and added comment on user home list of providers to describe more fully the owner identifier to use when uploading. - Altered method of pagination in styled sru search results. - Rationalised the display of data sources. - Created administration datasource edit screen. - Added Solr index restore to dpp/admin - Added version number to / and /dpp ------------------------------------------------------------------- ISPP Aggregator 0.2 ------------------------------------------------------------------- - Correctly expire login cookie - Improved configuration of SRU server, property mappings now defined outside the service. - Upload errors now correctly reported in UI instead of failing silently. - Added search option to data provider home page - Added Request Permission functionality to user home page - Added adminstative functions to approve/deny permission requests. ------------------------------------------------------------------- ISPP Aggregator 0.1 -------------------------------------------------------------------