Commit 962f891b authored by Enrico Ros's avatar Enrico Ros
Browse files

For Users:

Display contents faster, waste less memory. Added preloading to forward-
generate pages (if threading enabled and memory profile >= normal). Main
pages are always generated first, then thumbnails come, then preload ones,
etc.. Btw memory and cpu will be happier now.
(1 bug in memory and 1 in preloading still remains.. will be spotted soon)

For Developers:
Generator: dropped moc, api changes, better functions naming.
PDFGenerator and Document: moved requests queue to the Document.
   The generator performs only a single request a time. A flag can be
   queried to know if the generator is ready for starting another pixmap
   generation or not (in case it's generating a pixmap in background).
PixmapRequest: added priority and backgound (async) attributes.
Document: queuing requests by priority in requestPixmaps.
Observer: added defines for priorities used in doc->requestpixmaps.
PageView: delayed find-ahead timer creation (not created if not needed).
   Added preload PixmapRequests to the pixmap requests list.
ThumbnailsList: adapted to pixmapRequest changes and changed naming for
   delayed thumbanils request.

svn path=/trunk/kdegraphics/kpdf/; revision=379820
parent c0029ac3
......@@ -6,24 +6,13 @@ Legend:
MRG - MeRGed from head
(*) - Some parts of this item are already done
In progress:
-> gen / mem: priorities for requests (pageview, thumbnails, pv preloading, th preloading)
-> mem: iterate from older pages to newest ones when freeing memory (not rand)
-> gen / mem: sync Memory Management (in Document) with the Generator, so
generators can be notified when cancelling jobs (use the notification signal
from generator) -> real cleaning
-> gen / mem: check for removing 'canUnloadPixmap' from observers in place of requesting a
list of visible pixmaps. Think at adding visible list to ObserverData and
passing the visible list from observers (to remove visi lists)
-> generator: add preloading
Urgent fixes and items to get ready before 3.4 (special high-priority list):
-> fix: find-as-you-type (start kpdf, /, aaa, backspace, open doc)
In progress:
-> new word highlighting for searches / other highlights (on paper - enrico)
-> cleanup code and update README.png
-> mem: iterate from older pages to newest ones when freeing memory (not rand)
More items (first items will enter 'In progress list' first):
-> cleanup code and update README.png
-> find-as-you-type: use shortcut for 'find next' action (not the default one)
-> show Viewport in ThumbnailsList (blended/contour)
-> Delay TOC (DocumentSynapsis) generation (and move it on thread)
......@@ -88,7 +77,9 @@ More items (first items will enter 'In progress list' first):
-> Albert: Read pdf specification and see if paths with length = 1 are allowed, in case they are allowed see how to fix 97131 without skipping paths with length = 1
Done (newest features come first):
-> type ahead search in pageview (type '/' then the word to search..)
-> ADD: page preloading
-> FIX: smarter memory management / prioritize queries
-> ADD: type ahead search in pageview (type '/' then the word to search..) (JakubS)
-> FIX: scroll page if the the searched string is not visible
-> FIX: use a global Viewport over the document (linked views, real link following, location restoring, etc)
-> FIX: wrong zoom buttons order (BR74248) (check consistancy with kdvi/kviewshell/kghostview/.. (not konq))
......
......@@ -53,7 +53,7 @@ class KPDFDocumentPrivate
// observers / requests stuff
QMap< int, class ObserverData* > observers;
//QValueList< PixmapRequest * > asyncRequestsQueue;
QValueList< PixmapRequest * > pixmapRequestsStack;
// timers (memory checking / info saver)
QTimer * memCheckTimer;
......@@ -122,7 +122,7 @@ bool KPDFDocument::openDocument( const QString & docFile )
KMimeType::Ptr mime = KMimeType::findByPath( docFile );
QString mimeName = mime->name();
if ( mimeName == "application/pdf" )
generator = new PDFGenerator();
generator = new PDFGenerator( this );
else if ( mimeName == "application/postscript" )
kdError() << "PS generator not available" << endl;
else
......@@ -130,9 +130,6 @@ bool KPDFDocument::openDocument( const QString & docFile )
kdWarning() << "Unknown mimetype '" << mimeName << "'." << endl;
return false;
}
// get notification of completed jobs
connect( generator, SIGNAL( contentsChanged( int, int ) ),
this, SLOT( slotGeneratedContents( int, int ) ) );
// 1. load Document (and set busy cursor while loading)
QApplication::setOverrideCursor( waitCursor );
......@@ -181,6 +178,13 @@ void KPDFDocument::closeDocument()
delete generator;
generator = 0;
// remove requests left in queue
QValueList< PixmapRequest * >::iterator sIt = d->pixmapRequestsStack.begin();
QValueList< PixmapRequest * >::iterator sEnd = d->pixmapRequestsStack.end();
for ( ; sIt != sEnd; ++sIt )
delete *sIt;
d->pixmapRequestsStack.clear();
// send an empty list to observers (to free their data)
foreachObserver( notifySetup( QValueVector< KPDFPage * >(), true ) );
......@@ -252,12 +256,12 @@ bool KPDFDocument::isOpened() const
const DocumentInfo * KPDFDocument::documentInfo() const
{
return generator ? generator->documentInfo() : NULL;
return generator ? generator->generateDocumentInfo() : NULL;
}
const DocumentSynopsis * KPDFDocument::documentSynopsis() const
{
return generator ? generator->documentSynopsis() : NULL;
return generator ? generator->generateDocumentSynopsis() : NULL;
}
const KPDFPage * KPDFDocument::page( uint n ) const
......@@ -282,7 +286,7 @@ uint KPDFDocument::pages() const
bool KPDFDocument::okToPrint() const
{
return generator ? generator->allowed( Generator::Print ) : false;
return generator ? generator->isAllowed( Generator::Print ) : false;
}
QString KPDFDocument::getMetaData( const QString & key, const QString & option ) const
......@@ -290,35 +294,66 @@ QString KPDFDocument::getMetaData( const QString & key, const QString & option )
return generator ? generator->getMetaData( key, option ) : QString();
}
void KPDFDocument::requestPixmaps( const QValueList< PixmapRequest * > & requests, bool async )
void KPDFDocument::requestPixmaps( const QValueList< PixmapRequest * > & requests )
{
if ( !generator )
if ( !generator || requests.isEmpty() )
return;
#ifndef NDEBUG
//TODO REMOVE THIS
if ( !d->pixmapRequestsStack.isEmpty() && (*d->pixmapRequestsStack.begin())->priority == 0 )
kdDebug() << "calling requestPixmaps when SYNC generation has not yet finished." << endl;
#endif
// 1. [CLEAN STACK] remove previous requests of requesterID
int requesterID = requests.first()->id;
QValueList< PixmapRequest * >::iterator sIt = d->pixmapRequestsStack.begin(), sEnd = d->pixmapRequestsStack.end();
while ( sIt != sEnd )
{
if ( (*sIt)->id == requesterID )
sIt = d->pixmapRequestsStack.remove( sIt );
else
++sIt;
}
// 2. [ADD TO STACK] add requests to stack
bool threadingDisabled = !Settings::enableThreading();
bool preloadDisabled = threadingDisabled ||
Settings::memoryLevel() == Settings::EnumMemoryLevel::Low;
QValueList< PixmapRequest * >::const_iterator rIt = requests.begin(), rEnd = requests.end();
for ( ; rIt != rEnd; ++rIt )
{
// set the 'page field' (see PixmapRequest) and check if request is valid
// set the 'page field' (see PixmapRequest) and check if it is valid
PixmapRequest * request = *rIt;
request->page = pages_vector[ request->pageNumber ];
if ( !request->page || request->page->width() < 1 || request->page->height() < 1 )
if ( !(request->page = pages_vector[ request->pageNumber ]) )
continue;
// 1. Update statistics (pageMemory / totalMemory) adding this pixmap
ObserverData * obs = d->observers[ request->id ];
int pageNumber = request->pageNumber;
if ( obs->pageMemory.contains( pageNumber ) )
obs->totalMemory -= obs->pageMemory[ pageNumber ];
int pixmapMemory = 4 * request->width * request->height / 1024;
obs->pageMemory[ pageNumber ] = pixmapMemory;
obs->totalMemory += pixmapMemory;
if ( !request->async )
request->priority = 0;
// 2. Perform pre-cleaning if needed
mCleanupMemory( request->id );
if ( request->async && threadingDisabled )
request->async = false;
// 3. Enqueue to Generator (that takes ownership of request)
generator->requestPixmap( request, Settings::enableThreading() ? async : false );
// add request to the 'stack' at the right place
if ( !request->priority )
// add priority zero requests to the top of the stack
d->pixmapRequestsStack.push_back( request );
else
{
// insert in stack sorted by priority
sIt = d->pixmapRequestsStack.begin();
sEnd = d->pixmapRequestsStack.end();
while ( sIt != sEnd && (*sIt)->priority >= request->priority )
++sIt;
d->pixmapRequestsStack.insert( sIt, request );
}
}
// 3. [START FIRST GENERATION] if generator is ready, start a new generation,
// or else (if gen is running) it will be started when the new contents will
//come from generator (in requestDone())
if ( generator->canGeneratePixmap() )
sendGeneratorRequest();
}
void KPDFDocument::requestTextPage( uint page )
......@@ -329,7 +364,7 @@ void KPDFDocument::requestTextPage( uint page )
// Memory management for TextPages
generator->requestTextPage( kp );
generator->generateSyncTextPage( kp );
}
/* REFERENCE IMPLEMENTATION: better calling setViewport from other code
void KPDFDocument::setNextPage()
......@@ -600,6 +635,58 @@ bool KPDFDocument::print( KPrinter &printer )
return generator ? generator->print( printer ) : false;
}
void KPDFDocument::requestDone( PixmapRequest * req ) //FIXME
{
// notify an observer that its pixmap changed
if ( d->observers.contains( req->id ) )
d->observers[ req->id ]->instance->notifyPageChanged( req->pageNumber, DocumentObserver::Pixmap );
// delete request
delete req;
// start a new generation
#ifndef NDEBUG
if ( !generator->canGeneratePixmap() )
kdDebug() << "requestDone with generator not in READY state." << endl;
#endif
sendGeneratorRequest();
}
void KPDFDocument::sendGeneratorRequest()
{
// find a request
PixmapRequest * request = 0;
while ( !d->pixmapRequestsStack.isEmpty() && !request )
{
PixmapRequest * r = d->pixmapRequestsStack.last();
d->pixmapRequestsStack.pop_back();
// request only if page isn't already present
if ( !r->page->hasPixmap( r->id, r->width, r->height ) )
request = r;
else
delete r;
}
// if no request found (or already generated), return
if ( !request )
return;
// FIXME CHECK PREALLOC
// MEM: calc memory that should be freed to allow the new generation
int pixmapMemory = 4 * request->width * request->height / 1024;
// MEM: update statistics counting this pixmap
ObserverData * obs = d->observers[ request->id ];
int pageNumber = request->pageNumber;
if ( obs->pageMemory.contains( pageNumber ) )
obs->totalMemory -= obs->pageMemory[ pageNumber ];
obs->pageMemory[ pageNumber ] = pixmapMemory;
obs->totalMemory += pixmapMemory;
// MEM: cleanup
mCleanupMemory( request->id );
// submit the request to the generator
generator->generatePixmap( request );
}
void KPDFDocument::mCleanupMemory( int observerId )
{
......@@ -906,13 +993,6 @@ void KPDFDocument::slotCheckMemory()
mCleanupMemory( it.key() /*observerId*/ );
}
void KPDFDocument::slotGeneratedContents( int id, int pageNumber )
{
// notify an observer that its pixmap changed
if ( d->observers.contains( id ) )
d->observers[ id ]->instance->notifyPageChanged( pageNumber, DocumentObserver::Pixmap );
}
/** DocumentViewport **/
......@@ -1060,4 +1140,3 @@ DocumentSynopsis::DocumentSynopsis()
}
#include "document.moc"
#include "generator.moc"
......@@ -73,7 +73,7 @@ class KPDFDocument : public QObject // only for a private slot..
// perform actions on document / pages
void setViewportPage( int page, int id = -1 );
void setViewport( const DocumentViewport & viewport, int id = -1 );
void requestPixmaps( const QValueList< PixmapRequest * > & requests, bool asyncronous );
void requestPixmaps( const QValueList< PixmapRequest * > & requests );
void requestTextPage( uint page );
bool findText( const QString & text = "", bool caseSensitive = false, bool findAhead = false );
void findTextAll( const QString & pattern, bool caseSensitive );
......@@ -82,11 +82,15 @@ class KPDFDocument : public QObject // only for a private slot..
bool print( KPrinter &printer );
void unHilightPages(bool filteredOnly = true);
// notifications sent by generator
void requestDone( PixmapRequest * request );
signals:
void linkFind();
void linkGoToPage();
private:
void sendGeneratorRequest();
// memory management related functions
void mCleanupMemory( int observerId );
int mTotalMemory();
......@@ -98,14 +102,12 @@ class KPDFDocument : public QObject // only for a private slot..
void processPageList( bool documentChanged );
Generator * generator;
QString documentFileName;
QValueVector< KPDFPage * > pages_vector;
class KPDFDocumentPrivate * d;
private slots:
void saveDocumentInfo() const;
void slotCheckMemory();
void slotGeneratedContents( int id, int pageNumber );
};
......
......@@ -13,18 +13,15 @@
#include <qobject.h>
#include <qvaluevector.h>
#include <qstring.h>
#include "core/document.h"
class KPrinter;
class KPDFPage;
class KPDFLink;
class KPDFDocument;
class DocumentSynopsis;
class DocumentInfo;
class PixmapRequest;
/* Note: on contents generation and asyncronous queries.
* Many observers may want to request data syncronously or asyncronously.
* - Sync requests. These should be done in-place. Syncronous events in the
* queue have precedence on all the asyncronous ones.
* - Sync requests. These should be done in-place.
* - Async request must be done in real background. That usually means a
* thread, such as QThread derived classes.
* Once contents are available, they must be immediately stored in the
......@@ -44,32 +41,41 @@ class PixmapRequest;
*/
class Generator : public QObject
{
Q_OBJECT
public:
/** virtual methods to reimplement **/
// load a document and fill up the pagesVector
virtual bool loadDocument( const QString & fileName, QValueVector< KPDFPage* > & pagesVector ) = 0;
virtual bool loadDocument( const QString & fileName, QValueVector< KPDFPage * > & pagesVector ) = 0;
// Document description and Table of contents
virtual const DocumentInfo * documentInfo() { return 0L; }
virtual const DocumentSynopsis * documentSynopsis() { return 0L; }
virtual const DocumentInfo * generateDocumentInfo() { return 0L; }
virtual const DocumentSynopsis * generateDocumentSynopsis() { return 0L; }
// DRM handling
enum Permissions { Modify = 1, Copy = 2, Print = 4, AddNotes = 8 };
virtual bool allowed( int /*permisisons*/ ) { return true; }
virtual bool isAllowed( int /*permisisons*/ ) { return true; }
// generator core
virtual bool print( KPrinter& /*printer*/ ) { return false; }
virtual void requestPixmap( PixmapRequest * request, bool asyncronous ) = 0;
virtual void requestTextPage( KPDFPage * page ) = 0;
// page contents generation
virtual bool canGeneratePixmap() = 0;
virtual void generatePixmap( PixmapRequest * request ) = 0;
virtual void generateSyncTextPage( KPDFPage * page ) = 0;
// check configuration and return true if something changed
// print document using already configured kprinter
virtual bool print( KPrinter& /*printer*/ ) { return false; }
// access meta data of the generator
virtual QString getMetaData( const QString &/*key*/, const QString &/*option*/ ) { return QString(); }
// tell generator to re-parse configuration and return true if something changed
virtual bool reparseConfig() { return false; }
// Access meta data of the generator
virtual QString getMetaData( const QString &/*key*/, const QString &/*option*/ ) { return QString(); }
/** 'signals' to send events the KPDFDocument **/
// tell the document that the job has been completed
void signalRequestDone( PixmapRequest * request ) { m_document->requestDone( request ); }
signals:
void contentsChanged( int id, int pageNumber );
/** constructor: takes the Document as a parameter **/
Generator( KPDFDocument * doc ) : m_document( doc ) {};
private:
Generator();
KPDFDocument * m_document;
};
/**
......@@ -77,18 +83,24 @@ class Generator : public QObject
*/
struct PixmapRequest
{
// public data fields
PixmapRequest( int rId, int n, int w, int h, int p, bool a = false )
: id( rId ), pageNumber( n ), width( w ), height( h ),
priority( p ), async( a ), page( 0 ) {};
// observer id
int id;
// page number and size
int pageNumber;
int width;
int height;
// this field is set by the document before passing the
// asyncronous request priority (less is better, 0 is max)
int priority;
// generate the pixmap in a thread and notify observer when done
bool async;
// this field is set by the Docuemnt prior passing the
// request to the generator
KPDFPage * page;
// public constructor: initialize data
PixmapRequest( int rId, int n, int w, int h )
: id( rId ), pageNumber( n ), width( w ), height( h ), page( 0 ) {};
};
#endif
......@@ -36,12 +36,12 @@
#include "core/pagetransition.h"
#include "conf/settings.h"
// id for DATA_READY ThreadedGenerator Event
// id for DATA_READY PDFPixmapGeneratorThread Event
#define TGE_DATAREADY_ID 6969
PDFGenerator::PDFGenerator()
: pdfdoc( 0 ), kpdfOutputDev( 0 ),
docInfoDirty( true ), docSynopsisDirty( true )
PDFGenerator::PDFGenerator( KPDFDocument * doc )
: Generator( doc ), pdfdoc( 0 ), kpdfOutputDev( 0 ), ready( true ),
pixmapRequest( 0 ), docInfoDirty( true ), docSynopsisDirty( true )
{
// generate kpdfOutputDev and cache page color
reparseConfig();
......@@ -55,15 +55,8 @@ PDFGenerator::~PDFGenerator()
if ( generatorThread )
{
generatorThread->wait();
if ( !generatorThread->isReady() )
generatorThread->endGeneration();
delete generatorThread;
}
// remove requests in queue
QValueList< PixmapRequest * >::iterator rIt = requestsQueue.begin(), rEnd = requestsQueue.end();
for ( ; rIt != rEnd; rIt++ )
delete *rIt;
requestsQueue.clear();
// remove other internal objects
docLock.lock();
delete kpdfOutputDev;
......@@ -72,6 +65,7 @@ PDFGenerator::~PDFGenerator()
}
//BEGIN Generator inherited functions
bool PDFGenerator::loadDocument( const QString & fileName, QValueVector<KPDFPage*> & pagesVector )
{
#ifndef NDEBUG
......@@ -133,7 +127,7 @@ bool PDFGenerator::loadDocument( const QString & fileName, QValueVector<KPDFPage
}
const DocumentInfo * PDFGenerator::documentInfo()
const DocumentInfo * PDFGenerator::generateDocumentInfo()
{
if ( docInfoDirty )
{
......@@ -173,7 +167,7 @@ const DocumentInfo * PDFGenerator::documentInfo()
return &docInfo;
}
const DocumentSynopsis * PDFGenerator::documentSynopsis()
const DocumentSynopsis * PDFGenerator::generateDocumentSynopsis()
{
if ( !docSynopsisDirty )
return &docSyn;
......@@ -199,142 +193,84 @@ const DocumentSynopsis * PDFGenerator::documentSynopsis()
return &docSyn;
}
void PDFGenerator::addSynopsisChildren( QDomNode * parent, GList * items )
bool PDFGenerator::canGeneratePixmap()
{
int numItems = items->getLength();
for ( int i = 0; i < numItems; ++i )
{
// iterate over every object in 'items'
OutlineItem * outlineItem = (OutlineItem *)items->get( i );
return ready;
}
// 1. create element using outlineItem's title as tagName
QString name;
Unicode * uniChar = outlineItem->getTitle();
int titleLength = outlineItem->getTitleLength();
for ( int j = 0; j < titleLength; ++j )
name += uniChar[ j ];
if ( name.isEmpty() )
continue;
QDomElement item = docSyn.createElement( name );
parent->appendChild( item );
void PDFGenerator::generatePixmap( PixmapRequest * request )
{
#ifndef NDEBUG
if ( !ready )
kdDebug() << "calling generatePixmap() when not in READY state!" << endl;
#endif
// update busy state (not really needed here, because the flag needs to
// be set only to prevent asking a pixmap while the thread is running)
ready = false;
// 2. find the page the link refers to
LinkAction * a = outlineItem->getAction();
if ( a && a->getKind() == actionGoTo )
{
// page number is contained/referenced in a LinkGoTo
LinkGoTo * g = static_cast< LinkGoTo * >( a );
LinkDest * destination = g->getDest();
if ( !destination && g->getNamedDest() )
{
// no 'destination' but an internal 'named reference'. we could
// get the destination for the page now, but it's VERY time consuming,
// so better storing the reference and provide the viewport as metadata
// on demand
item.setAttribute( "ViewportName", g->getNamedDest()->getCString() );
}
else if ( destination->isOk() )
{
// we have valid 'destination' -> get page number
int pageNumber = destination->getPageNum() - 1;
if ( destination->isPageRef() )
{
Ref ref = destination->getPageRef();
pageNumber = pdfdoc->findPage( ref.num, ref.gen ) - 1;
}
// set page as attribute to node
// TODO add other attributes to the viewport
item.setAttribute( "Viewport", DocumentViewport( pageNumber ).toString() );
}
}
// debug message
kdDebug() << "id: " << request->id << " is requesting " << (request->async ? "ASYNC" : "sync") << " pixmap for page " << request->page->number() << " [" << request->width << " x " << request->height << "]." << endl;
// 3. recursively descend over children
outlineItem->open();
GList * children = outlineItem->getKids();
if ( children )
addSynopsisChildren( &item, children );
/** asyncronous requests (generation in PDFPixmapGeneratorThread::run() **/
if ( request->async )
{
// start the generation into the thread
generatorThread->startGeneration( request );
return;
}
}
void PDFGenerator::addTransition( int pageNumber, KPDFPage * page )
{
Page *pdfPage = pdfdoc->getCatalog()->getPage( pageNumber + 1 );
if ( !pdfPage )
return;
/** syncronous request: in-place generation **/
// compute dpi used to get an image with desired width and height
KPDFPage * page = request->page;
double fakeDpiX = request->width * 72.0 / page->width(),
fakeDpiY = request->height * 72.0 / page->height();
PageTransition *pdfTransition = pdfPage->getTransition();
if ( !pdfTransition || pdfTransition->getType() == PageTransition::Replace )
return;
// setup kpdf output device: text page is generated only if we are at 72dpi.
// since we can pre-generate the TextPage at the right res.. why not?
bool genTextPage = !page->hasSearchPage() && (request->width == page->width()) &&
(request->height == page->height());
// generate links and image rects if rendering pages on pageview
bool genRects = request->id == PAGEVIEW_ID;
KPDFPageTransition *transition = new KPDFPageTransition();
switch ( pdfTransition->getType() ) {
case PageTransition::Replace:
// won't get here, added to avoid warning
break;
case PageTransition::Split:
transition->setType( KPDFPageTransition::Split );
break;
case PageTransition::Blinds:
transition->setType( KPDFPageTransition::Blinds );
break;
case PageTransition::Box:
transition->setType( KPDFPageTransition::Box );
break;
case PageTransition::Wipe:
transition->setType( KPDFPageTransition::Wipe );
break;
case PageTransition::Dissolve:
transition->setType( KPDFPageTransition::Dissolve );
break;
case PageTransition::Glitter:
transition->setType( KPDFPageTransition::Glitter );
break;
case PageTransition::Fly:
transition->setType( KPDFPageTransition::Fly );
break;
case PageTransition::Push:
transition->setType( KPDFPageTransition::Push );
break;
case PageTransition::Cover:
transition->setType( KPDFPageTransition::Cover );
break;
case PageTransition::Uncover:
transition->setType( KPDFPageTransition::Uncover );
break;
case PageTransition::Fade: